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[NAME OF DOCUMENT] SPECIFICATION 

[TITLE OF THE INVENTION] APPARATUS, METHOD AND PROGRAM FOR 
CONTROLLING EDITING IMAGE DISPLAY 

[NAME OF DOCUMENT] SCOPE OF CLAIMS 

[CLAIM 1] An apparatus for controlling an editing image display, 
characterized by comprising: 

a material capture means for capturing material data; 

an edit processing means for recording the material data captured by the 
material capture means and performing edit processing on the recorded material 
data; 

an edited-outputting signal generation measns for generating a 
outputting signal based on the material data recorded in the edit processing means 
or the material data being subjected to the edit processing and outputting the 
outputting signal; 

a user interface unit for generating an operation signal according to an 
editing operation; and 

an editing control means for controlling working of the material capture 
means, the edit processing means, and the edited outputting signal generation 
means based on the operation signal, 

wherein the editing control means controls the edited outputting signal 
generation means, to cause the edited outputting signal generation means to 
output the transmission signal for displaying the material data in a manner that 
allows items of the material data which are combined with frame rate information 
as associated information and items of the material data which are not combined 
with the the associated information to be distinguished from each other. 

[CLAIM 2] The apparatus according to Claim 1, characterized in that the 
editing control means includes a material administration representation for 
displaying a list of items of the material data, and displays the material data in a 
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manner that allows the items of the material data which are combined with the 
associated information and the items of the material data which are not combined 
with the associated information to be distinguished from each other in the 
material administration representation. 

[CLAIM 3] The apparatus according to Claim 1, characterized in that the 
editing control means includes a reproduction order representation for arranging 
items of the material data in a reproduction order to produce a content, and 
displays the material data in a manner that allows the items of the material data 
which are combined with the associated information and the items of the material 
data which are not combined with the associated information to be distinguished 
from each other in the reproduction order representation. 

[CLAIM 4] The apparatus according to Claim 1, characterized in that the 
editing control means includes a reproduction time order representation for 
allocating the items of the material data along a time axis to produce a content, 
and displays the material data in a manner that allows the items of the material 
data which are combined with the associated information and the items of the 
material data which are not combined with the associated information to be 
distinguished from each other in the reproduction order representation. 

[CLAIM 5] The apparatus according to Claim 1, characterized in that the 
editing control means sets a speed range available for reproduction of the 
material data based on the associated information, and controls the edited 
outputting signal generation means to cause the edited outputting signal 
generation means to output the outputting signal for displaying the set speed 
range available for reproduction. 

[CLAIM 6] The apparatus according to Claim 5, characterized in that the 
editing control means includess a material administration representation and 
displaying a list of items of the material data, and displays the set speed range 
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avilable for reproduction in the material administration representation. 

[CLAIM 7] The apparatus according to Claim 5, characterized in that the 
editing control means includes a reproduction order representation for arranging 
items of the material data in a reproduction order to produce a content, and 
displays the set speed range avilable for reproduction in the reproduction order 
representation. 

[CLAIM 8] The apparatus according to Claim 5, characterized in that the 
editing control means includes a reproduced image representation of the material 
data, and displays the set speed range avilable for reproduction in the reproduced 
image representation. 

[CLAIM 9] The apparatus according to Claim 8, characterized in that the 
control means includes, within a representation of the speed range available for 
reproduction, an indication for indicating a reproduction speed. 

[CLAIM 10] The apparatus according to Claim 5, characterized in that 
the editing control means includes a reproduction time order representation for 
allocating items of the material data along a time axis to produce a content, and 
displays a representation width of the material data being altered according to 
reproduction time calculated on the basis of the reproduction speed in the 
reproduction time order representation. 

[CLAIM 1 1] The apparatus according to Claim 5, characterized in that 
the editing control means includes a reproduced image representation of the 
material data and a reproduction time order representation for allocating items of 
the material data along a time axis to produce a content, displays the set speed 
range available for reproduction in the reproduced image representation, includes, 
within a representation of the speed range available for reproduction, an 
indication for indicating a reproduction speed, and displays a representation 
width of the material data being altered according to reproduction time calculated 
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on the basis of the reproduction speed in the reproduction time order 
representation, 

and in that when the operation means alters the representation width of 
the material data in the reproduction time order representation, a display position 
of the indication is moved according to the operation; and when the display 
position of the indication is altered, the representation width of the material data 
in the reproduction time order representation is altered according to the operation. 

[CLAIM 12] The apparatus according to Claim 5, characterized in that, 
when the material data is set to be reproduced at a reproduction speed within the 
speed range available for reproduction, the editing control means controls the 
edited outputting signal generation means to cause the edited outputting signal 
generation means to output a outputting signal based on the material data 
reproduced at the set reproduction speed to reproduce the material data at the set 
reproduction speed. 

[CLAIM 13] A method for controlling an editing image display, 
characterized by comprising the steps of: 

capturing material data; 

performing an edit processing on the captured material data; and 
displaying, in the edit processing, an image based on the captured 
material data or material data being subjected to the edit processing, 

wherein the material data is displayed in the edited image in a manner 
that allows items of the material data which are combined with frame rate 
information as associated information and items of the material data which are 
not combined with the the frame rate information to be distinguished from each 
other. 

[CLAIM 14] The method according to Claim 13, characterized in that a 
material administration representation for displaying a list of items of the material 
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data is provided, and the material data is displayed in a manner that allows the 
items of the material data that are combined with the associated information and 
the items of the material data that are not combined with the associated 
information to be distinguished from each other in the material administration 
representation. 

[CLAIM 15] The method according to Claim 13, characterized in that a 
reproduction order representation for arranging items of the material data in a 
reproduction order to produce a content is provided, and the material data is 
displayed in a manner that allows the items of the material data that are combined 
with the associated information and the items of the material data that are not 
combined with the associated information to be distinguished from each other in 
the reproduction order representation. 

[CLAIM 16] The method according to Claim 13, characterized in that a 
reproduction time order representation for allocating items of the material data 
along a time axis to produce a content is provided, and the material data is 
displayed in a manner that allows the items of the material data that are combined 
with the associated information and the items of the material data that are not 
combined with the associated information to be distinguished from each other in 
the reproduction time order representation. 

[CLAIM 17] The method according to Claim 13, characterized in that a 
speed range available for reproduction of the material data is set based on the 
associated information, and the set speed range available for reproduction is 
displayed. 

[CLAIM 18] The method according to Claim 17, characterized in that a 
material administration representation for displaying a list of items of the material 
data is provided, and the set speed range avilable for reproduction is displayed in 
the material administration representation. 
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[CLAIM 19] The method according to Claim 17, characterized in that a 
reproduction order representation for arranging items of the material data in a 
reproduction order to produce a content is provided, and the set speed range 
avilable for reproduction is displayed in the reproduction order representation. 

[CLAIM 20] The method according to Claim 17, characterized in that a 
reproduced image representation of the material data is provided, and the set 
speed range avilable for reproduction is displayed in the reproduced image 
representation. 

[CLAIM 21] The method according to Claim 20, characterized in that an 
indication for indicating a reproduction speed is provided within a representation 
of the speed range available for reproduction,. 

[CLAIM 22] The method according to Claim 17, characterized in that a 
reproduction time order representation for allocating items of the material data 
along a time axis to produce a content is provided and a representation width of 
the material data being altered according to reproduction time calculated on the 
basis of the reproduction speed is displayed in the reproduction time order 
representation. 

[CLAIM 23] The method according to Claim 17, characterized in that a 
reproduced image representation of the material data and a reproduction time 
order representation for allocating items of the material data along a time axis to 
produce a content are provided, the set speed range available for reproduction is 
displayed in the reproduced image representation, an indication for indicating a 
reproduction speed is provided within a representation of the speed range 
available for reproduction, and a representation width of the material data being 
altered according to reproduction time calculated on the basis of the reproduction 
speed is displayed in the reproduction time order representation, 

and in that when the operation means alters the representation width of 
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the material data in the reproduction time order representation, a display position 
of the indication is moved according to the operation; and when the display 
position of the indication is altered, the representation width of the material data 
in the reproduction time order representation is altered according to the operation. 

[CLAIM 24] The method according to Claim 17, characterized in that 
when the material data is set to be reproduced at a reproduction speed within the 
speed range available for reproduction, a signal based on the material data 
reproduced at the set reproduction speed is output to reproduce the material data 
at the set reproduction speed. 

[CLAIM 25] A program for allowing a computer to carry out a method 
for controlling an editing image display, the method comprising the steps of: 

setting, when frame rate information is obtained from material data to be 
edited, a speed available for reproduction based on the frame rate information; 

performing display control according to the frame rate information; 

displaying, when a reproduction speed is set, the set reproduction speed; 

calculating, based on the set reproduction speed, reproduction time in the 
case of reproducing the material data at the set reproduction speed; 

performing display control according to the reproduction time; and 

displaying reproduced images of the material data at the indicated 
reproduction speed. 

[DETAILED DESCRIPTION OF THE INVENTION] 
[0001] 

[TECHNICAL FIELD TO WHICH THE INVENTION BELONGS] 
The invention relates to an apparatus, a method, and program for 
controlling an editing image display. Particularly, an edit processing is 
performed using captured material data. In the edit processing, an image based 
on the captured material data or a material data being subjected to the edit 
processing is displayed. The material data is displayed in the edited image in a 
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manner that allows items of the material data that are combined with frame rate 
information as associated information and items of the material data which are 
not combined with the the frame rate information to be distinguished from each 
other. Further, display control based on the combined frame rate information is 
performed. 

[0002] 

[PRIOR ART] 

From the past, in generation of contents of image or image audio used 
for broadcasting, a distribution via a communication network, circulation by 
recording medium, or the like, a content having its motion speed altered has been 
used in addition to a content generated at a predetermined reference frame rate in 
order to obtain effects intended by a creator. 

[0003] 

In generation of such content having its motion speed altered, a higher 
frame rate as compared to the reference frame rate is set to generate a content, for 
example, and the content is reproduced at the reference frame rate to thereby 
produce a content with a low motion speed. Alternatively, a lower frame rate as 
compared to the reference frame rate is set to generate a content, and the content 
is reproduced at the reference frame rate to thereby produce a content with a high 
motion speed. Further, by adjusting the frame rate set for generation of the 
content or the frame rate for reproduction of the content, the motion speed of the 
content can be freely altered. 

[0004] 

Thus, by performing an edit processing using the content having its 
motion speed altered and the content generated at the reference frame rate, a 
content that is capable of obtaining an effect intended by a creator is generated. 

[0005] 

It has been proposed in Patent Document 1 that the video camera that is 
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capable of compressing or extending a time axis in order to generate a content 
having its frame rate altered. 
[0006] 

[Patent Document] 

Japanese Patent Application Laid-open No. HEI 11-177930 
[0007] 

[PROBLEMS TO BE SOLVED BY THE INVENTION] 
Incidentally, in a case of performing the edit processing using the 
content generated at the reference frame rate and the content having its motion 
speed altered as material as described above, it is not easily determined which 
frame rate is used to generate each content, by merely displaying an image of the 
content. Further, in a case of performing a speed change processing on the 
content having its motion speed altered, it is difficult to determine whether or not 
the speed change processing can be easily performed without causing any 
deterioration in image quality. If the frame rate that is set (hereinafter, referred 
to as, "set frame rate") is 10 times the reference frame rate, when the content 
generated at the set frame rate is reproduced at the reference frame rate, a motion 
speed of the content becomes one-tenth. Here, it is conceivable that since the 
set frame rate is 10 times the reference frame rate, when the creator wants to 
reproduce the content at a one-fifth motion speed, the speed change can be easily 
performed without any deterioration in image quality by performing a 
frame-skipping for each frame. However, it is difficult to determine whether or 
not the speed change can be easily performed without any deterioration in image 
quality, based on merely the displayed image. . 
[0008] 

In this regard, the present invention provides an apparatus and method 
for controlling an editing image display that are capable of easily performing an 
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edit processing using a material generated at the reference frame rate and a 
material having its frame rate altered. 
[0009] 

[MEANS FOR SOLVING THE PROBLEMS] 

According to the present invention, there is provided an apparatus for 
controlling an editing image display including: a material capture means for 
capturing material data; an edit processing means for recording the material data 
captured by the material capture means and performing edit processing on the 
recorded material data; an edited outputting signal generation measns for 
generating an outputting signal based on the material data recorded in the edit 
processing means or the material data being subjected to the edit processing and 
outputting the outputting signal; a user interface unit for generating an operation 
signal according to an editing operation; and an editing control means for 
controlling working of the material capture means, the edit processing means, and 
the edited outputting signal generation means based on the operation signal, 
wherein the editing control means controls the edited outputting signal generation 
means, to cause the edited outputting signal generation means to output the 
transmission signal for displaying the material data in a manner that allows items 
of the material data which are combined with frame rate information as 
associated information and items of the material data which are not combined 
with the the associated information to be distinguished from each other. 

[0010] 

A method for controlling an editing image display includes: capturing 
material data; performing an edit processing on the captured material data; and 
displaying, in the edit processing, an image based on the captured material data or 
material data being subjected to the edit processing, wherein the material data is 
displayed in the edited image in a manner that allows items of the material data 
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which are combined with frame rate information as associated information and 
items of the material data which are not combined with the the frame rate 
information to be distinguished from each other. 
[0011] 

A program for allowing a computer to carry out a method for controlling 
an editing image display, said method including the steps of: setting, when frame 
rate information is obtained from material data to be edited, a speed available for 
reproduction based on the frame rate information; performing display control 
according to the frame rate information; displaying, when a reproduction speed is 
set, the set reproduction speed; calculating, based on the set reproduction speed, 
reproduction time in the case of reproducing the material data at the set 
reproduction speed; performing display control according to the reproduction 
time; and displaying reproduced images of the material data at the indicated 
reproduction speed. 

[0012] 

In this invention, material data is captured and an edit processing is 
performed using the captured material data. In the edit processing, an image 
based on the captured material data or a material data being subjected to the edit 
processing is displayed. The material data is displayed in the edited image in a 
manner that allows items of the material data that are combined with frame rate 
information as associated information and items of the material data which are 
not combined with the the frame rate information to be distinguished from each 
other. In the edited image, a material administration representation for 
displaying a list of items of the material data is provided, and the material data is 
displayed in a manner that allows the items of the material data that are combined 
with the associated information and the items of the material data that are not 
combined with the associated information to be distinguished from each other in 
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the material administration representation. Further, a reproduction order 
representation for arranging items of the material data in a reproduction order to 
produce a content is provided, and the material data is displayed in a manner that 
allows the items of the material data that are combined with the associated 
5 information and the items of the material data that are not combined with the 

associated information to be distinguished from each other in the reproduction 
order representation. Further, a reproduction time order representation for 
allocating items of the material data along a time axis to produce a content is 
provided, and the material data is displayed in a manner that allows the items of 

1 0 the material data that are combined with the associated information and the items 

of the material data that are not combined with the associated information to be 
distinguished from each other in the reproduction time order representation. 
Further, a speed range available for reproduction of the material data is set based 
on the associated information, and the set speed range available for reproduction 

15 is displayed. This speed range available for reproduction is displayed in the 

material administration representation for displaying a list of items of the material 
data, the reproduction order representation for arranging items of the material 
data in a reproduction order to produce a content, and the reproduced image 
representation of the material data. An indication for indicating a reproduction 

2 0 speed is provided within a representation of the speed range available for 

reproduction. The reproduction time order representation for allocating items of 
the material data along a time axis to produce a content is provided and a 
representation width of the material data being altered according to reproduction 
time calculated on the basis of the reproduction speed is displayed in the 

2 5 reproduction time order representation. Further, when an operation that alters 

the representation width of the material data in the reproduction time order 
representation, a display position of the indication is moved according to the 
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operation; and when the display position of the indication is altered, the 
representation width of the material data in the reproduction time order 
representation is altered according to the operation. 
[0013] 

5 [EMBODIMENT MODE OF THE INVENTION] 

Hereinafter, a description will be given on an embodiment of the present 
invention. Fig. 1 shows an entire configuration of a content-editing system for 
an image content and/or an audio content. An imaging apparatus 10 generates 
image data having its frame rate altered and combines associated information 
1 0 including frame rate information indicating a frame rate of the image data with 

this image data to supply an editing apparatus 30 with the combined ones as 
material data DTm. When the imaging apparatus 1 0 is provided with an audio 
input apparatus 20, the imaging apparatus 10 generates audio data to supply the 
editing apparatus 30 with the audio data together with the associated information 
15 as material data DTm. The material data DTm may also be supplied from an 

apparatus other than the imaging apparatus 10. 
[0014] 

The editing apparatus 30 performs edit processing on the supplied 
material data DTm to produce and output content-data DC. The editing 

2 0 apparatus 30 also produces an image signal Sym concerning the editing and 

supplies an edited-image-displaying apparatus 40 with it, to thereby check a 
progress, result and the like of the edit processing of an image through a 
displayed image on the edited-image-displaying apparatus 40. Similarly, the 
editing apparatus 30 produces an audio signal Sam concerning the editing and 

2 5 supplies an edited-audio-outputting apparatus 41 with it to thereby check a 

progress, result and the like of the edit processing of an audio through an audio 
output from the editing-audio-transmitting apparatus 41. 
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[0015] 

Fig. 2 shows a configuration of the imaging apparatus 10. Light that is 
incident through an imaging lens system 1 1 enters an imaging unit 12, and an 
object image is formed on an image surface of an image pickup device such as a 
charge coupled device (CCD) provided within the imaging unit 12. The image 
pickup device generates imaging charges of the object image through 
photoelectric conversion. In addition, the generated imaging charges are read 
out based on a driving signal CR from a timing generator 142, which will be 
described later, and an imaging signal Sp having a frame rate that corresponds to 
the driving signal CR is generated to be supplied to a camera processing circuit 
131 in a signal processor 13. 

[0016] 

Based on a timing signal CT received from the timing generator 142, the 
camera processing circuit 131 performs various signal processes at timings 
synchronized with the imaging signal Sp. The signal processes include a 
process for reducing noise components from the imaging signal Sp by means of 
correlated dual sampling or the like, a process for converting the noise-reduced 
imaging signal Sp into digital image data, a clamp process of the image data, 
shading correction, pilling-up of deficiency in the image pickup device, y 
correction, a contour calibration process, and a knee correction process, for 
example. Further, the camera processing circuit 1 3 1 performs other various 
signal processes under the process conditions based on an operation control 
signal CS received from an imaging control circuit 141 of a controller 14. As 
described above, an image data DV obtained through various signal processes by 
the camera processing circuit 131 is supplied to an outputting unit 15. 

[0017] 

The timing generator 142 of the controller 14 generates the driving 
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signal CR according to the operation control signal CS from the imaging control 
circuit 141 and supplies the imaging unit 12 with the driving signal CR, thereby 
altering a reading-out frequency of the imaging charges in the imaging unit 12 to 
control a frame rate of the imaging signal Sp to be set to a set frame rate FRs 
5 based on an operation signal PSa from a user interface unit 1 6. For example, a 

frame frequency of a reference frame rate FRr is set to 59.94Hz or 29.97Hz in an 
NTSC system, and 50Hz or 25Hz in a PAL system. When an operation of 
setting the set frame rate FRs to the n times the reference frame rate FRr, the 
frame rate of the imaging signal Sp is controlled to be equal to the n times the 
1 0 reference frame rate FRr. 

[0018] 

The timing generator 142 also generates the timing signal CT 
synchronized with the driving signal CR and supplies the camera processing 
circuit 13 1 and the audio processing circuit 132 with the timing signal CT. The 
15 timing generator 142 further generates associated information DM including the 

set frame rate FRs that is the frame rate of the image data DV as frame rate 
information, and supplies the outputting unit 15 with the associated information 
DM. 

[0019] 

2 0 The imaging control circuit 141 of the controller 14 is connected with 

the user interface unit 16. When the imaging apparatus 10 performs a switching 
operation and a frame rate alteration operation, the user interface unit 16 
generates an operation signal PSa according to these operations and supplies the 
imaging control circuit 141 with the operation signal PSa. When receiving the 

2 5 operation signal PSa from external equipment, the user interface unit 16 also 

supplies the imaging control circuit 141 with the operation signal PSa. 
[0020] 
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The imaging control circuit 141 generates the operation control signal 
CS based on the operation signal PSa received from the user interface unit 16 
such that the imaging apparatus 10 operates according to the operation signal PSa, 
and supplies the camera processing circuit 13 1 and the timing generator 142 with 
5 the operation control signal CS. 

[0021] 

An audio processing circuit 132 is supplied with an analog audio signal 
Sa from the audio input apparatus 20. The audio processing circuit 132 also 
performs a sampling process on the analog audio signal Sa based on the timing 
1 0 signal CT received from the timing generator 142 to generate digital audio data 

DA and supply the outputting unit 1 5 with the digital audio data DA. 
[0022] 

The outputting unit 15 combines the image data DV and the audio data 
DA with the associated information DM to generate material data DTm and 
1 5 supplies the editing apparatus 30 with the material data DTm. When the 

material data DTm or a recording signal produced on the basis of the material 
data DTm is recorded on recording medium, the material data DTm may be 
supplied to the editing apparatus 30 through the recording medium by 
reproducing the material data DTm or the recording signal produced on the basis 
2 0 of the material data DTm recorded on the recording medium in the editing 

apparatus 30. 

[0023] 

Here, in the case of combining the image data DY and the audio data DA 
with the associated information DM, when the image data DV and the audio data 
25 DA are compressed to generate the material data DTm as a data stream, the 

associated information DM is inserted into the data stream of image or a header 
of the data stream. 
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[0024] 

Further, when an SDI format standardized as SMPTE (Society of Motion 
Picture and Television Engineers) 259M (Television- 10-Bit 4:2:2 Component and 
4fsc Composite Digital Signals-Serial Digital Interface) is used to transfer 
non-compressed image and audio data, or when an SDTI format standardized as 
SMPTE 305M (Television-Serial Data Transport Interface (SDTI)) or an 
SDTI-CP format standardized as SMPTE 326M (Television- SDTI Content 
Package Format (SDTI-CP)) obtained by further defining the SDTI format is 
used to transfer compressed image and audio data, the associated information 
DM is inserted into a signal having each format as UMID data standardized as 
SMPTE 330M(Television-Unique Material Identifier (UMID)). 

[0025] 

Incidentally, the imaging apparatus 10 described above alters the 
read-out timing of the imaging charges in the imaging unit 12 in order to obtain 
the imaging signal Sp having the set frame rate FRs. However, even when the 
read-out timing of the imaging charges in the imaging unit 12 is not altered, the 
imaging signal Sp having the set frame rate FRs may be generated. That is, the 
image data DV having the set frame rate FRs can be generated by generating 
image data DVa having a constant frame rate higher than the set frame rate FRs 
and extracting from the image data DVa only a portion of image data 
corresponding to the set frame rate FRs. Fig. 3 shows a configuration in such a 
case. In Fig. 3, parts corresponding to those of Fig. 2 are denoted by the same 
reference numerals, and the detailed description thereof is omitted. 

[0026] 

A timing generator 182 in a controller 18 generates a driving signal CRa 
according to the maximum value of the set frame rate FRs that is set through the 
user interface unit 16 and supplies the imaging unit 12 with the driving signal 
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CRa. Based on the driving signal CRa, the imaging unit 12 generates an 
imaging signal Spa having a fixed frame rate FRq higher than the reference frame 
rate FRs. The imaging unit 1 2 then supplies the camera processing circuit 1 3 1 
in a signal processor 17 with the imaging signal Spa. When the set frame rate 
5 FRs may be altered up to n times the reference frame rate FRr, the imaging unit 

12 generates the imaging signal Spa having a frame rate n times the reference 
frame rate FRr, and supplies the camera processing circuit 131 with the imaging 
signal Spa. 

[0027] 

1 0 The timing generator 1 82 also generates a timing signal CTa 

synchronized with the driving signal CRa and supplies the camera processing 
circuit 131 and the audio processing circuit 132 in the signal processor 17 and an 
effective frame signal generation circuit 183 with the timing signal CTa. 
[0028] 

1 5 The camera processing circuit 1 3 1 supplied the image data DVa having 

the fixed frame rate FRq generated based on the imaging signal Spa to an 
effective data selection circuit 171. The audio signal processing circuit 1 32 
supplies an audio data DAa obtained by carrying out a sampling based on the 
timing signal CTa having a fixed frequency to the effective data selection circuit 

20 171. 

[0029] 

An imaging control circuit 181 generates a set information signal CF 
indicating the set frame rate FRs based on the operational signal PSa from the 
user interface unit 16 and supplies the effective frame signal generation circuit 
25 1 83 with the set information signal CF. 

[0030] 

The effective frame signal generation circuit 183 extracts data on a 
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frame basis from the image data DVa based on a ratio of the frame rate FRq of 
the image data DVa previously fixed to a predetermined value to the set frame 
rate FRs indicated in the set information signal CF and generates an extraction 
control signal CC for generating image data DV having the set frame rate FRs. 
The effective frame signal generation circuit 183 also synchronizes the extraction 
signal CC with the timing signal CTa and supplies the effective data selection 
circuit 171 with the synchronized extraction control signal CC. For example, 
when the frame rate FRq of the image data DVa is n times the reference frame 
rate FRr and the set frame rate FRs is n/2 times the reference frame rate FRr, the 
effective frame signal generation circuit 1 83 generates the extraction control 
signal CC for performing data extraction on a frame basis every other frame from 
the image data DVa and supplies the effective data selection circuit 171 with the 
extraction control signal CC synchronized with the timing signal CTa. The 
effective frame signal generation circuit 183 further generates the associated 
information DM having the set frame rate FRs as the frame rate information 
based on the set information signal CF and supplies the outputting unit 15 with 
the associated information DM. 
[0031] 

The effective data selection circuit 171 extracts the image data DVa and 
the audio data DAa of the frame indicated by the extraction control signal CC and 
supplies the outputting unit 15 with them as the image data DV and the audio 
data DA. In addition, although not shown, the effective frame signal generation 
circuit 183 may supply the effective data selection circuit 171 with the associated 
information DM including the set frame rate FRs as the frame rate information, 
and the effective data selection circuit 171 may perform frame-skipping on the 
audio data DAa according to a ratio of the set frame rate FRs to a frame rate 
when the audio data DAa is generated. In a case where the frame rate FRq 
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when the audio data DAa is generated is n times the reference frame rate FRr and 
the set frame rate FRs is n/2 times the reference frame rate FRr, the effective data 
selection circuit 171 performs the frame-skipping on the audio data DAa every 
other sample. In this case, since an interval of the frame-skipping may be 
shortened as compared to a case of performing the frame-skipping on the audio 
data DAa on a frame basis, audio having an excellent sound quality may be 
obtained based on the audio data DA. 
[0032] 

Thus, the image data DVa having a fixed frame frequency makes 
unnecessary the alteration of operation frequencies in the imaging unit 12 and the 
camera processing circuit 1 3 1 of the signal processor 1 7. This allows 
configurations of the imaging unit 12 and the camera processing circuit 131 to be 
made simpler. Since the image data DV having the set frame rate FRs can be 
generated by only extracting data on a frame basis from the image data DVa, the 
image data DV having a desired set frame rate FRs can be easily generated from 
the image data DVa. 

[0033] 

Further, the imaging apparatus may be provided with a video memory or 
an adder and a divider to generate the image data DV by adding the image data 
every predetermined frames. In this case, a variable range of a frame rate of the 
imaging signal Sp can be made narrower. That is, by adding the imaging signal 
Sp of n frames and dividing the signal level by n, the signal having a frame rate 
1/n times the imaging signal Sp can be obtained even when a frame rate of the 
imaging signal Sp is not divided by n. 

[0034] 

Figs. 4 and 5 are diagrams showing a relationship between the image 
data DV generated by the imaging apparatuses 10, 10a and the associated 
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information DM. When the set frame rate FRs is set to the one equal to or twice 
the reference frame rate FRr as shown in Fig. 4A, the associated information DM 
that includes the frame rate information DM-FRs indicating the set frame rate 
FRs, as shown in Fig. 4C, is combined with the image data DV as shown in Fig. 
4B (in the figures, frame images based on the image data DV are shown). Fig. 
4D illustrates a relationship between time and the frame images. The frame rate 
information DM-FRs may include a magnification of the set frame rate FRs with 
respect to the reference frame rate FRr in addition to the set frame rate FRs. Fig. 
4C and subsequent figures show the frame rate information DM-FRs together 
with the magnification. 
[0035] 

When the set frame rate FRs is set to the one equal to or half as much as 
the reference frame rate FRr as shown in Fig. 5A, the associated information DM 
that includes the frame rate information DM-FRs indicating the set frame rate 
FRs, as shown in Fig. 5C is combined with image data DV as shown in Fig. 5B 
(in Fig. 5B, frame images based on the image data DV are shown). Fig. 5D 
illustrates a relationship between time and the frame images. 

[0036] 

Next, a description will be given on the editing apparatus 30. Fig. 6 
shows a configuration of the editing apparatus 30. The material data DTm 
supplied to the editing apparatus 30 is supplied to an information detection circuit 
3 1 1 in a material capture unit 31. The information detection circuit 3 1 1 detects 
the associated information DM from the material data DTm. The detected 
associated information DM is supplied to a database-processing circuit 312. 
The database-processing circuit 312 also is also supplied with image data DV and 
audio data DA included in the material data DTm. 

[0037] 
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The database-processing circuit 312 records the image data DV, the 
audio data DA, and the associated information DM detected in the information 
detection circuit 3 1 1 on a data recording apparatus 321 in an edition processing 
unit 32 with the image data DV and the audio data DA being linked to the 
associated information DM. The database-processing circuit 312 generates 
database information DB that allows contents of material data to be easily 
checked on the basis of the associated information DM and the image data DV 
and the audio data DA linked to the associated information DM, which have been 
recorded on the data recording apparatus 321. The generated database 
information DB is then supplied to an editing control unit 33. The database 
information DB is constituted of information including information for allowing 
content of material data to be recognized (for example, thumbnails), a time length 
of the material data, a set frame rate FRs, and a recorded location on the data 
recording apparatus 32 1 , for example. 

[0038] 

The editing control unit 33 generates image data DVg for allowing edit 
processing to be carried out under graphical user interface (GUI) environment 
and image data DVi for allowing representation of content of the database 
information. The editing control unit 33 then supplies an image-outputting 
signal generation circuit 351 with the image data DVg and DVi. The 
image-outputting signal generation circuit 351 generates an image signal Svm 
based on the supplied image data DVg, DVi and outputs it to the 
edited-image-displaying apparatus 40. Thus, supplying the 
edited-image-displaying apparatus 40 with the image signal Svm allows 
information on what kind of the material data is recorded or the like to be 
displayed on a screen of the edited-image-displaying apparatus 40. 

[0039] 
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The editing control unit 33 also controls post-production process. 
Specifically, when an operational signal PSe that utilizes representation under 
GUI environment is supplied from a user interface unit 34 connected with the 
editing control unit 33 and the operational signal PSe indicates the selection of 
5 any kind of the material data, the editing control unit 33 generates a reading 

control signal RC according to the operational signal Pse and supplies a 
write/read processing circuit 322 in the edition processing unit 32 with the 
reading control signal RC. When the operational signal PSe relates to an editing 
operation such as processing and linkage of the read material data, the editing 

1 0 control unit 33 generates an editing control signal ET according to the operational 

signal PSe and supplies a signal editing circuit 323 in the edition processing unit 
32 with the editing control signal ET. When editing of the material data is 
finished to bring content-data to completion, the editing control unit 33 generates 
a writing control signal WC according to the operational signal PSe if the 

1 5 operational signal PSe indicates recording operation of the content-data on the 

data recording apparatus 321 . The editing control unit 33 then supplies a 
write/read processing circuit 322 with the writing control signal WC. The 
editing control unit 33 also generates an outputting control signal RP according to 
the operational signal PSe when the operational signal PSe indicates data 

2 0 transmission of the content-data, and supplies the write/read processing circuit 

322 with the outputting control signal RP. The editing control unit 33 also 
generates a speed range setting signal LP according to the operational signal PSe 
when the operational signal PSe specifies a reproduction speed range of the 
content-data, supplies the signal editing circuit 323 with the speed range setting 

25 signal LP. 

[0040] 

Based on the reading control signal RC, the write/read processing circuit 
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322 reads required material data out of the data recording apparatus 321 to supply 
the signal editing circuit 323 with it. Based on the writing control signal WC, 
the write/read processing circuit 322 also writes the complete content-data on the 
data recording apparatus 321 . Based on the outputting control signal RP, the 
write/read processing circuit 322 reads required content-data DC out of the data 
recording apparatus 321 to output it. 
[0041] 

The signal editing circuit 323 performs edit processing such as 
processing, linkage, and deletion of image and audio data based on the editing 
control signal ET using the image data DV and the audio data DA included in the 
material data read out of the data recording apparatus 321. In this case, the 
signal editing circuit 323 supplies the image-outputting signal generation circuit 
3 5 1 with the image data DVe before, during or after the editing thereof. The 
signal editing circuit 323 also supplies the audio-outputting signal generation 
circuit 352 with the audio data DAe before, during or after the editing thereof 
When frame rates of the image data DV and the audio data DA are altered 
according to the edit processing, the signal editing circuit 323 also alters the 
associated information DM so as to correspond to the edited image data DV and 
the audio data DA. The signal editing circuit 323 further combines associated 
information DMc corresponding to the edited image data DV and the edited audio 
data DA with the edited image data DV and the edited audio data DA to generate 
the content-data DC. When the signal editing circuit 323 receives the speed 
range setting signal LP, the signal editing circuit 323 also combines speed range 
information indicating the reproduction speed range of the content-data DC as the 
associated information DMc based on the speed range setting signal LP. When 
the signal editing circuit 323 receives information on a title of the content and a 
recommended reproduction speed of the content from the user interface unit 34, 
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the signal editing circuit 323 also combines the information as the associated 
information DMc. When the signal editing circuit 323 receives information on a 
reproduction time length of the content-data according to the edit processing, the 
signal editing circuit 323 combines this information as the associated information 
5 DMc. Further, when the signal editing circuit 323 receives information on the 

maximum reproduction speed available for reproduction of the content-data, the 
signal editing circuit 323 combines the information on the maximum 
reproduction speed as the associated information DMc. 
[0042] 

1 0 The image-outputting signal generation circuit 35 1 of an edited 

outputting signal generation unit 35 generates an image signal Svm based on the 
image data DVg, DVi supplied from the editing control unit 33 and outputs it to 
the edited-image-displaying apparatus 40, as described above. This allows 
information on the material data to be displayed under GUI environment. 

1 5 Further, by generating the image signal Svm based on the image data DVe 

supplied from the signal editing circuit 323, the image before, during or after the 
editing thereof can be checked on the screen of the edited-image-displaying 
apparatus 40. 

[0043] 

2 0 The audio-outputting signal generation circuit 352 converts the audio 

data DAe supplied from the signal editing circuit 323 to an analog audio signal 
Sam and brings it to a desired signal level to supply the edited-audio-outputting 
apparatus 41 constituted of a speaker, a head hone, and the like with the 
converted signal. This allows the audio before, during or after the editing 

2 5 thereof to be checked through an audio output from the 

editing-audio-transmitting apparatus 41. 
[0044] 



Application No.2002-332650 



As described above, when the content-data Dc is complete in the editing 
apparatus 30 through the post-production process using the material data DTm, 
the editing apparatus 30 outputs the complete content-data DC. 

[0045] 

5 Next, a description will be given on operations of the editing apparatus 

30. Fig. 7 illustrates a GUI screen displayed on the edited-image-displaying 
apparatus 40 for an editing operation. In the GUI screen, a material 
administration browser window 401 for displaying a list of items of the material 
data as a material administration representation is provided at an upper-left side 

1 0 thereof; a story board window 402 for arranging items of the material data in a 

reproduction order to produce a content as a reproduction order representation is 
provided at an lower-left side thereof; a monitor viewer window 403 for 
displaying an image of the material data before or after the edit processing as a 
reproduced image representation is provided at an upper-right side thereof; and a 

1 5 time line window 404 for allocating items of the material data along a time axis 

to produce a content as a reproduction time order representation is provided at a 
lower-left side thereof. An operation control portion 405 is provided between 
the monitor viewer window 403 and the time line window 404. It should be 
noted that locations and sharps of the material administration browser window 

2 0 401 , the story board window 402, the monitor viewer window 403, the time line 

window 404, and the operation control portion 405 are respectively illustrative 
ones and thus, of course, they are not limited to the above locations and shapes. 
[0046] 

The material administration browser window 401 shows a list of items of 
2 5 the material data stored in the data recording apparatus 321. For each item of 

the material data, a stamp view (a thumbnail view) for indicating a title, a length, 
and content of an item of the stored material data, a speed range available for 
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reproduction at which a natural image may be reproduced without noise, and the 
like are shown. 
[0047] 

The story board window 402 serves as a working area for production of 
5 a content. The story board window 402 arranges items of the material data on a 

reproduced order to produce a content. The story board window 402 also 
displays a speed range available for reproduction. The monitor viewer window 
403 displays not only an image based on the material data but also a variable 
speed bar for indicating a speed range available for reproduction and an 
1 0 indication of a reproduction speed within the speed range. 

[0048] 

The time line window 404 serves as a working area for allocating items 
of the material data along a time axis to perform a more detailed operation for 
production of content. The time line window 404 also alters a representation 
1 5 width of the material data according to reproduction time calculated based on the 

reproduction speed. The operation control portion 405 displays operation keys 
for reproducing the items of material data and the contents arranged on the time 
line window 404. 

[0049] 

2 0 Based on a displayed GUI image and the operational signal PSe received 

from the user interface unit 34, the editing control unit 33 determines whether or 
not a drag-and-drop operation, a key-in operation in the operation control portion 
405, and the like are carried out. If the editing control unit 33 determines that 
the drag-and-drop operation and the like have been carried out, the editing control 

2 5 unit 33 controls operations of the editing apparatus 30 according to such 

drag-and-drop operation and the like. 
[0050] 
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Fig. 8 is a flowchart showing a display control at the editing operation. 
At step ST11, the editing apparatus 30 determines whether or not the processing 
of the material data is requested. If the processing of the material data is not 
requested, the process goes back to the step ST1 1 . If the processing of the 
5 material data is requested, the process goes to step ST12. 

[0051] 

At step ST 12, the editing apparatus 30 determines whether or not frame 
rate information DM-FRs indicating the set frame rate FRs is obtained from the 
material data to be edited. Here, the editing apparatus 30 refers to the database 

1 0 information DB, and when it is determined that the frame rate information 

DM-FRs indicating the set frame rate FRs is not obtained from the material data, 
the process goes to the step ST 13 where the editing apparatus 30 performs 
display control so that the material data has the same reproduction speed as a 
normal reproduction speed. When the editing apparatus 30 determines that the 

1 5 frame rate information DM-FRs is obtained from the material data, the process 

then goes to step ST14. 
[0052] 

At step ST14, a speed available for reproduction is set based on the set 
frame rate FRs indicated by the frame rate information DM-FRs. This speed is 

2 0 set so that a noise-less natural reproduction image can be realized through a 

frame-skipping of a predetermined frame period or a frame repetition. For 
example, if a multiple of the set frame rate FRs to the reference frame rate FRr is 
more than one, the speed available for reproduction is set by searching for 
common divisors of this multiple other than one and calculating reciprocals of the 

2 5 searched common divisors. Specifically, if the set frame rate FRs is 1 0 times 

the reference frame rate FRr, the common divisors of this multiple other than one 
are 2, 4, 5, and 10 and thus, the reciprocals thereof are 1/2, 1/4, 1/5, and 1/10, 
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respectively. By selecting one of the reciprocals of the divisors, the speed 
available for reproduction can be set to less than the same reproduction speed as 
normal reproduction speed. When the speed available for reproduction not less 
than the same reproduction speed as the normal reproduction speed is set to 
positive integer multiples of the normal reproduction speed, the frame-skipping 
periods in each of the speeds available for reproduction may be equal to each 
other. An upper limit of the speed available for reproduction is set to a speed at 
which content of the material data can be grasped easily. 
[0053] 

If a multiple of the set frame rate FRs to the reference frame rate FRr is 
less than one, the speed available for reproduction is set by searching for the 
reciprocals of this multiple and calculating common multiples of the searched 
reciprocals other than one or integer multiples of the searched reciprocals. If the 
set frame rate FRs is 1/6 times the reference frame rate FRr, the divisors of the 
searched reciprocals other than one are of 2, 3, and 6 and the integer multiples of 
the reciprocals are of 6, 12, 1 8, — , respectively. By selecting one of the divisors 
of the reciprocals and the integer multiples of the reciprocals as the speed 
available for reproduction, frame-skipping periods or numbers of the frame 
repetitions in each of the speeds available for reproduction may be equal to each 
other. Further, in a case where the speed is set to less than the same speed as the 
normal reproduction speed when the multiple of the set frame rate FRs is smaller 
that one, the resultant image is a repetition of frame images. Thus, a lower limit 
of the speed available for reproduction may be set to the same reproduction speed 
as the normal reproduction speed. 

[0054] 

At step ST15, the editing apparatus 30 performs display control 
corresponding to the frame rate information DM-FRs, namely, to display so that 
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it can be determined that an item of the material data is combined with the frame 
rate information DM-FRs indicating the set frame rate FRs, as the associated 
information, and to display the speed range available for reproduction set based 
on the set frame rate FRs. For example, in the material administration browser 
5 window 401 and the story board window 402, in order to easily determine that 

the item of the material data is combined with the frame rate information 
DM-FRs, contours, colors, and the like of frames of stamp views 401a, 402a are 
made different from those of an item the material data that is not combined with 
the frame rate information DM-FRs. In addition, when an 

1 0 information-representing regions 40 1 b, 402b each for representing information 

on the material are provided, the speeds available for reproduction set in step 
ST14 are displayed in each of the regions. For example, in Fig.7 and Figs. 9, 
which will be described later, when the item of the material data is combined with 
the frame rate information DM-FRs, the frame having its width enlarged is 

15 displayed and the speed range available for reproduction is indicated as being 

"xO.l through x2.0". 
[0055] 

In displaying the time line window 404, similar to the cases of the 
material administration browser window 401 and the story board window 402, it 

2 0 is possible to easily determine that an item of the material data is combined with 

the frame rate information DM-FRs indicating the set frame rate FRs, as the 
associated information. For example, a contour, and a color of a frame of frame 
image representation region 404a are made different from those of an item of the 
material data that is not combined with the frame rate information DM-FRs. In 

2 5 the time line window 404, the frame image representation region 404a is adjusted 

along a horizontal direction, i.e., time axis direction thereof according to the set 
frame rate FRs. For example, if the set frame rate FRs is greater than the 
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reference frame rate FRr, a number of frame images per unit of time increases. 
Accordingly, a representation width of the frame image representation region 
404a is enlarged. If the set frame rate FRs is smaller than the reference frame 
rate FRr, a number of frame images per unit of time decreases. Accordingly, the 
representation width of the frame image representation region 404a is shortened. 
Further, the monitor viewer window 403 is provided with a reproduction speed 
representation 403 b and a variable-speed console representation 403c 
corresponding to the set speed available for reproduction in addition to a material 
image representation 403a based on the material data to display the reproduction 
image, the reproduction speed, and the speed available for reproduction. In 
addition, a cursor position of the variable-speed console representation 404c 
represented by a bold line allows a control direction to be determined. 
[0056] 

At step ST16, the editing apparatus 30 determines whether or not the 
reproduction speed is indicated via the user interface unit 34. When an 
operation for altering the representation width of the frame image representation 
region 404a is performed or when the cursor position in the variable-speed 
console representation 403 c represented in the monitor viewer window 403 is 
moved, the process goes to step ST1 7. When the reproduction speed is not 
indicated, the process goes back to step ST16. 

[0057] 

At step ST 17, the editing apparatus 30 determines the indicated 
reproduction speed to display the reproduction speed thus determined. For 
example, when an operation for altering the representation width of the frame 
image representation region 404a in the time line window 404 is performed, a 
reproduction speed is selected in sequence from the speeds available for 
reproduction set at step ST14 depending on the operational direction to set a 
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speed when completing the operation as the reproduction speed. When the 
cursor position in the variable-speed console representation 403c represented in 
the monitor viewer window 403 is moved, a reproduction speed is selected from 
the speeds available for reproduction set at step ST14 depending on the cursor 
5 position to set the selected speed as the reproduction speed. 

[0058] 

When the reproduction speed is determined, the indicated reproduction 
speed is shown in the reproduction speed representation 403b of the monitor 
viewer window 403. Further, the cursor position of the variable-speed console 

1 0 representation 403 c and the representation width of the frame image 

representation region 404a are being in cooperation with each other. Thus, 
when the reproduction speed is indicated by an operation of altering the 
representation width of the frame image representation region 404a, the cursor 
position of the variable-speed console representation 403c is moved to a position 

1 5 corresponding to the indicated reproduction speed to display the reproduction 

speed. When the reproduction speed is indicated by an operation of altering the 
cursor position in the variable-speed console representation 403c, the 
representation width of the frame image representation region 404a is altered to a 
width corresponding to the indicated reproduction speed. As described above, 

2 0 since the cursor position in the variable-speed console representation 403 c and 

the representation width of the frame image representation region 404a are being 
in cooperation with each other, the reproduction speed can be correctly altered 
using either of these two methods. 
[0059] 

25 At step ST1 8, based on the indicated reproduction speed, the editing 

apparatus 30 calculates reproduction time when reproducing the material data at 
the indicated reproduction speed. For example, when the set frame rate FRs is 
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10 times the reference frame rate FRr, a slow speed available for reproduction is 
of any one of "1/10, 1/5, 1/4 and 1/2" of the normal reproduction speed. For 
example, if the reproduction time of the material is 30 seconds when the 
reproduction speed is the same speed as normal reproduction speed, the 
reproduction time when the reproduction speed is 1/10 times the normal 
reproduction speed is 300 seconds. The reproduction time when the 
reproduction speed is 1/5 times the normal reproduction speed is 150 seconds. 
As described above, by multiplying the reproduction time when the reproduction 
speed is the same speed as normal reproduction speed by the reciprocal of the 
multiple of the reproduction speed at the reproduction, the reproduction time can 
be calculated. 

[0060] 

At step ST19, the editing apparatus 30 performs a display control 
according to the reproduction time. For example, the representation width of 
the frame image representation region 404a in the time line window 404 is altered 
according to the reproduction time calculated at step ST1 8, and the process goes 
back to step ST 16. During reproduction operation of the material data, the 
reproduced image when reproducing the material data at the indicated 
reproduction speed is displayed as the material image representation 403a of the 
monitor viewer window 403. 

[0061] 

If the reproduction time on a scene of a part of the material data is 
altered, the scene is split and then, the above process is conducted on the split 
scenes, thereby allowing alteration of the reproduction time. 

[0062] 

Figs.9 show an editing operation screen in the case of using the material 
data combined with the frame rate information DM-FRs indicating that, for 
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example, the set frame rate FRs is 10 times the reference frame rate FRr. Fig. 
9A illustrates the GUI screen where the reproduction speed is the same speed as 
the normal reproduction speed. Fig. 9B illustrates the GUI screen where the 
reproduction speed is one-tenth times the normal reproduction speed. Fig. 9C 
5 illustrates the GUI screen where the reproduction speed is twice as much as the 

normal reproduction speed. For example, in a case of an item of the material 
data that is combined with the information on the set frame rate FRs, a frame out 
of the item of the material data is displayed in the story board window 402 and 
the time line window 404 in a manner different from a case of an item of the 
1 0 material data that is not combined with the frame rate information DM-FRs 

indicating the set frame rate FRs. 
[0063] 

When reproducing the selected material data, the set frame rate FRs is 
determined on the basis of the frame rate information DM-FRs, and then, a 

15 reproduction speed FP is multiplied by the set frame rate FRs to calculate a 

determined value FD,and reproduction conditions are determined based on the 
determined value FD. For example, when the reproduction speed is the same 
speed as the normal reproduction speed as shown in Fig. 9A and the set frame 
rate FRs is 10 times the reference frame rate FRr, the determined value FD is 

2 0 ( 1 * 1 0= 1 0). It should be noted that Fig. 1 OA illustrates images based on the 

image data DV in a case where the set frame rate FRs is 10 times the reference 
frame rate FRr. Fig. 10B illustrates the frame rate information DM-FRs 
indicating the set frame rate FRs of each of the frame images. Fig. 1 0C 
illustrates the absolute frame numbers AN. 

25 [0064] 

When the determined value FD is set to 10, namely, FD=10, as shown in 
Figs. 10D and 10E, the image signal Svm is generated using every 10 frames, 
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namely, with the image data of nine frames being skipped to display reproduced 
images having the same speed as the normal reproduction speed on the monitor 
viewer window 403 based on the image signal Svm. It should be noted that Fig. 
10D illustrates the absolute frame numbers AN of the displayed frame images, 
5 and Fig. 10E illustrates frame images displayed on the material image 

representation 403a of the monitor viewer window 403 by the image signal Svm. 
[0065] 

Next, when an operation of enlarging the representation width of the 
frame image representation region 404a in the time line window 404 along a 

1 0 direction indicated by an arrow A or moving the cursor position in the 

variable-speed console representation 403c in a direction indicated by an arrow B 
is carried out, processing as the slow motion of reproduction is performed. For 
example, when the reproduction speed is set to 1/10 times the normal 
reproduction speed, the reproduction speed representation 403 b in the monitor 

1 5 viewer window 403 is altered to "x 0. 1 " as shown in Fig. 9B. At the same time, 

the cursor position in the variable-speed console representation 403c is moved to 
a cursor position corresponding to a speed of one-tenth times the normal 
reproduction speed. Further, since the reproduction time becomes longer, the 
representation width of the frame image representation region 404a is enlarged. 

2 0 [0066] 

If the reproduction speed is set to 1/10 times the normal reproduction 
speed as shown in the displayed screen of Fig. 9B, the determined value FD is set 
to one, namely, FD=10*(1/10)=1. When the determined value FD is set to one, 
namely, FD=1, the image signal Svm is generated using every FD=1 frame of the 

2 5 image data DV, namely, without any frame being skipped. This allows the 

reproduced images having a speed of one-tenth times the normal reproduction 
speed to be displayed on the material image representation 403a of the monitor 
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viewer window 403 as shown in Fig. 10A. 
[0067] 

When an operation of shortening the representation width of the frame 
image representation region 404a in the time line window 404 along the direction 
5 indicated by the arrow B or moving the cursor position in the variable-speed 

console representation 403c in the direction indicated by an arrow A is carried out, 
a processing as the quick motion of reproduction is performed. For example, 
when the reproduction speed is set to twice as much as the normal reproduction 
speed, the reproduction speed representation 403 b in the monitor viewer window 

1 0 403 is altered to "x 2.0" as shown in Fig. 9C. In addition, the cursor position in 

the variable-speed console representation 403c is moved to a cursor position 
corresponding to twice of the normal reproduction speed. Further, since the 
reproduction time becomes shorter, the representation width of the frame image 
representation region 404a is shortened, as shown in Fig. 9C. 

15 [0068] 

If the reproduction speed is set to twice of the normal reproduction speed 
as shown in the displayed screen of Fig. 9C, the determined value FD is set to 20, 
namely, FD=1 0*2=20. When the determined value FD is set to 20, namely, 
FD=20, the image signal Svm is generated using every 20 frames, namely, with 

2 0 the image data of nineteen frames being skipped. This allows the reproduced 

images having a twice speed as much as the normal reproduction speed to be 
represented on the monitor viewer window 403 based on the image signal Svm, 
as shown in Figs. 10F and 10G. It should be noted that Fig. 10F illustrates the 
absolute frame numbers AN of the displayed frame images, and Fig. 10G 

2 5 illustrates frame images displayed in the material image representation 403a of 

the monitor viewer window 403. 
[0069] 
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Thus, the noise-less slow reproduction may be easily realized by 
carrying out frame-skipping on a frame basis based on the frame rate information 
DM-FRs and the reproduction speed. It should be noted that by enabling an 
operation of altering the reproduction speed during the reproduction, the 
reproduced image can be checked at a desired reproduction speed at easy. 

[0070] 

Since the reproduction speed and the representation width are displayed 
with them being linked with each other, a relationship between the reproduction 
speed and expansion/contraction of the time axis may be easily grasped, thereby 
allowing the editing operation to be easily conducted. It should be noted that, 
when the frame image representation region 404a is provided with an indication 
for indicating a direction in which the representation width of the frame image 
representation region 404a can be altered, namely, a direction in which the 
reproduction speed can be altered, for example, an arrow indication illustrated by 
a dotted line, it is possible to find out the direction in which the reproduction 
speed can be altered, at easy. 

[0071] 

Fig. 1 1 is a flowchart showing an edit processing operation in which 
display control as shown in Fig. 8 is performed and the GUI screen as shown in 
Figs. 9 is used, for example. At step ST21 the material data is captured and 
recorded on the data recording apparatus 321 to generate the database 
information DB. 

[0072] 

At step ST22, content of the captured material data is displayed using the 
database information DB. Displaying the content of the captured material data 
is carried out by causing the material administration browser window 401 to 
display a stamp view and information of the captured material data. When any 
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material data displayed on the material administration browser window 401 is 
selected, the monitor viewer window 403 displays image of the selected material 
data. Further, when a reproduction control key representation in the operation 
control portion 405 provided between the monitor viewer window 403 and the 
time line window 404 is operated, reproduction, stop, and reproduction at various 
speeds or the like of the material data is carried out according to the key 
operation, and then the reproduced image is displayed in the monitor viewer 
window 403. 

[0073] 

At step ST23, whether or not selection of material data is made is 
determined. If the selection of material data is not made, the process goes back 
to step ST23. If the selection of material data is made, the process goes to step 
ST24. For example, when a drag-and-drop operation from the material 
administration browser window 401 to the time line window 404 is made, it is 
determined that the selection of material data is made, and the process goes to 
step ST24. 

[0074] 

At step ST24, the editing apparatus 30 arranges frame images generated 
based on the selected material data in an imaged time order and displays them on 
the frame image representation region 404a of the time line window 404. 

[0075] 

At step ST25, the material data is processed according to various editing 
operations such as cut and paste or replacement of the frame images displayed on 
the time line window 404. The editing apparatus 30 then generates a desired 
image and audio data to produce content-data. In addition, the associated 
information DMc indicating the set frame rate and the like is combined to the 
image and audio data to create the content-data. 
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[0076] 

At step ST26, a completion process of the content-data is performed, in 
which the content-data created at step ST25 is subjected to processes such as a 
process on bridges of the material data, addition enhancement of effects, and 
5 synthesis of images and audio to be added. At step ST27, an outputting process 

of the content-data is performed. That is, the complete content-data is output as 
program data for broadcasting to a program-sending apparatus. Alternatively, 
the complete content-data is output as content-data for distribution to a 
contents-sending apparatus. Further, an authoring process such as addition of 
1 0 information for carrying out special reproduction corresponding to a menu list for 

reproducing the content or a reproduction menu and the like to the complete 
content-data. Then, the data subjected to the authoring process is recorded on 
the recording medium. 
[0077] 

1 5 Incidentally, the above edit processing by the editing apparatus 30 may 

also be realized through a software process using a computer. Fig. 12 shows a 
configuration in a case where the edit processing is carried out using the 
software. 

[0078] 

2 0 The computer includes a CPU (Central Processing Unit) 381 therein as 

shown in Fig. 12. A ROM 382, a RAM 383, a data storage unit 384 constituted 
of a hard disk drive with a large storage capacity and the like, and an input/output 
interface 385 are connected with the CPU 381 via a bus 390. A signal-inputting 
unit 391, a signal-outputting unit 392, and a recording medium drive 393 are 

2 5 connected with the input/output interface 385. ■ 

[0079] 

The CPU 381 executes programs stored on the ROM 382, the RAM 383 
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or the data storage unit 384 to perform the edit processing as shown in Fig. 8 and 
Fig. 1 1 . Material data input to the signal-inputting unit 391 is stored on the data 
storage unit 384 via the input/output interface 385 and the bus 390. In addition, 
an item of the material data to be used for editing is read from the material data 
5 stored on the data storage unit 384, and the edit processing is performed on the 

item of the material data, and the complete content-data is stored on the data 
storage unit 384 again. The complete content-data stored on the data storage 
unit 384 is read out and output through the signal-outputting unit 392. 
[0080] 

10 It is conceivable that a program for performing the edit processing may 

be previously stored on the ROM 382 or the data storage unit 384. Alternatively, 
a program for performing contents-sending process stored on the recording 
medium or a program stored on the recording medium may be read out and 
executed. A program may be output through wired or wireless transmission line. 

15 The received program may be executed in the computer. 

[0081] 

Thus, according to the embodiment as described above, the items of the 
material data which are combined with the frame rate information as the 
associated information and the items which are not combined with the frame rate 

2 0 information is displayed in a manner that allows them to be distinguished from 

each other. Accordingly, a scene available for reproducing a slow motion image 
by using the frame rate information of the material data can be easily found out. 
Further, since a speed range available for reproduction is displayed on the basis 
of the associated information, a variable speed reproduction can be realized by 

2 5 setting a speed within the speed range. Accordingly, effective image expression 

can be easily enhanced. 
[0082] 
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The editing apparatus 30 may also provide the contents having a wide 
speed range available for reproduction using the material data effectively if it 
may generate the content-data without any frame rate of the material data being 
reduced. 

[0083] 

[EFFECT OF THE INVENTION] 

According to the invention, material data is captured and an edit 
processing is performed using captured material data. In the edit processing, an 
image based on the captured material data or a material data being subjected to 
the edit processing is displayed. The material data is displayed in the edited 
image in a manner that allows items of the material data that are combined with 
frame rate information as associated information and items of the material data 
which are not combined with the the frame rate information to be distinguished 
from each other. Accordingly, a scene available for reproducing a slow motion 
image by using the frame rate information of the material data can be easily 
found out. 

[0084] 

Further, a speed range available for reproduction of the material data is 
set based on the associated information, and the determined speed range available 
for reproduction is displayed. Accordingly, the reproduction speed at which 
reproduction can be performed without noise can be easily set. In addition, in a 
reproduction time order representation for allocating items of the material data 
along a time axis to produce a content, a representation width of the material data 
is altered according to reproduction time calculated on the basis of the 
reproduction speed in said reproduction time order representation. Accordingly, 
expansion/contraction of the time axis can be grasped visually. Further, when 
an operation of altering the representation width of the material data in the 
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reproduction time order representation is performed, an indication of the 
reproduction speed is altered according to the operation. When an operation of 
altering the indication is performed, the representation width of the material data 
in the reproduction time order representation is altered according to the operation. 
Thus, the relation between the reproduction speed and the expansion/contraction 
of the time axis can be grasped easily. 

[BRIEF DESCRIPTION OF THE DRAWINGS] 

[FIGURE 1] A diagram for illustrating a whole configuration of a 
content-editing system; 

[FIGURE 2] A block diagram showing a configuration of an imaging apparatus; 

[FIGURE 3] A block diagram showing another configuration of the 
imaging apparatus; 

[FIGURE 4] Diagrams showing relationship (part 1) between image data 
and associated information; 

[FIGURE 5] Diagrams showing relationship (part 2) between the image 
data and the associated information; 

[FIGURE 6] A block diagram showing a configuration of an editing 
apparatus; 

[FIGURE 7] An illustration of GUI screen at an editing operation; 
[FIGURE 8] A flowchart showing a display control at the editing 
operation; 

[FIGURE 9] Illustrations each for GUI screen at the editing operation; 
[FIGURE 10] Illustrations showing a reproduction operation of the 

image; 

[FIGURE 11] A flowchart showing an edit processing operation; and 
[FIGURE 12] An illustration showing a configuration of a 
content-editing system for editing content by means of software. 
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[DESCRIPTION OF REFERENCE NUMERALS] 
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mm [NAME OF DOCUMENT] DRAWINGS 

imi] FIGURE 1 

1 0 Imaging apparatus 

2 0 Audio input apparatus 

3 0 Editing apparatus 

4 0 Edited-image-displaying apparatus 
4 1 Editing-audio-outputting apparatus 
= > 7- 1/ y BM ~^Ti Content-editing system 

[12] FIGURE 2 

1 0 Imaging apparatus 

1 1 Imaging lens system 

1 2 Imaging unit 

1 3 Signal processor 

1 4 Controller 

1 5 Outputting unit 

1 6 User interface unit 

2 0 Audio input apparatus 
13 1 Camera processing circuit 

13 2 Audio processing circuit 

14 1 Imaging control circuit 
14 2 Timing generator 
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[0 5] FIGURE 5 

VkW7 s —?ktt^¥fr<nM& (*:<D2) Relationship (part 2) between the 

image data and the associated information 
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[EI 6] FIGURE 6 

3 0 Editing apparatus 

3 1 Material capture unit 

3 2 Edition processing unit 

3 3 Editing control unit 

3 4 User interface unit 

3 5 Edited outputting signal generation unit 

4 0 Edited-image-displaying apparatus 
4 1 Editing-audio-outputting apparatus 
3 11 Information detection circuit 

3 12 Database-processing circuit 

3 2 1 Data recording apparatus 

3 2 2 Write/read processing circuit 
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FIGURE 8 
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Perform display control so that material data 
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Display determination of reproduction speed 
Calculate reproduction time 
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FIGURE 9 

GUI screen at editing operation 
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FIGURE 10 

Reproduction operation of image 

FIGURE 11 
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[NAME OF DOCUMENT] ABSTRACT 
[SUMMARY] 

[OBJECT] To easily perform edit processing using material generated at a 
standard frame rate and material generated at an altered frame rate. 
[SOLVING MEANS] When an image based on captured material data or material 
data being subjected to edit processing is displayed on a material administration 
browser 401, a story board window 402 and a time line window 404, items of the 
material data which are combined with frame rate information as associated 
information and items of the material data which are not combined with the frame 
rate information are displayed in a manner that allows them to be distinguished 
from each other. When displaying the items of the material data which are 
combined with the frame rate information as the associated information, a speed 
range available for reproduction that is set based on based the frame rate 
information of the material data is displayed. A monitor viewer window 403 
includes a reproduction speed representation 403b and a variable-speed console 
representation 403 c for altering a reproduction speed within the speed range 
available for reproduction. Reproduction images are displayed on a material 
image representation 403a at the set reproduction speed. 
[SELECTED DRAWING] Figure 7 
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|NAME OF DOCUMENT] SPECIFICATION 

[TITLE OF THE INVENTION] APPARATUS FOR CONTROLLING CONTENTS, METHOD 
AND PROGRAM FOR CONTROLLING EDITING IMAGE DISPLAY OF CONTENS 

[NAME OF DOCUMENT] SCOPE OF CLAIMS 

[CLAIM 1] An apparatus for controlling an editing image display comprising: 

a material capture means for capturing material data; 

a edit processing means for recording the material data captured by said material 
capture means and editing the recorded material data; 

a edit-transmitting signal generation measns for generating a transmitting signal 
based on the material data by said edit processing means or the material data on the 
editing process and transmitting; 

a user interface unit for generating operation signal corresponding to operations; 

and 

a editing control means for controling working of said material capture means, 
said edit processing means, and said edit-transmitting signal generation means based on 
the operation signal, 

wherein said editing control means controls said edit-transmitting signal 
generation means, and transmits a transmitting signal to display items of said material data 
that is combined with the frame rate information as the associated information and not 
combined with the frame rate information as the associated information with said items of 
the material data being distinguished from each other. 

[CLAIM 2] The apparatus according to Claim 1, wherein said editing control 
means provides a material administration representation for displaying a list of items of 
the material data, and displays said items of the material data that is combined with the 
associated information and not combined with the associated information with said items 
of the material data being distinguished from each other in the material administration 
representation. 
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[CLAIM 3] The apparatus according to Claim 1, wherein said editing control 
means provides a reproduction order representation to arrange in a reproduction order to 
create the contents, and displays said items of the material data that is combined with the 
as the associated information and not combined with the the associated information with 
5 said items of the material data being distinguished from each other in the reproduction 

order representation. 

[CLAIM 4] The apparatus according to Claim 1, wherein said editing control 
means provides a reproduction time order representation to allocate said the material data 
along a time axis to create the contents, and displays said items of the material data that is 
1 0 combined with the as the associated information and not combined with the the associated 

information with said items of the material data being distinguished from each other in the 
reproduction order representation. 

[CLAIM 5] The apparatus according to Claim 1, wherein said editing control 
means sets a speed range available for reproduction on said material data based on said 
15 associated information, and controls said edit-transmitting signal generation measns to 

transmit the transmitting signal for displaying speed range available for reproduction. 

[CLAIM 6] The apparatus according to Claim 5, wherein said editing control 
means provides a material administration representation for displaying a list of items of 
the material data and said speed range avilable for reproduction is displayed in said 
2 0 material administration representation. 

[CLAIM 7] The apparatus according to Claim 5, wherein said editing control 
means provides a reproduction order representation to arrange in a reproduction order to 
create the contents, and displays said determined speed range avilable for reproduction in 
the reproduction order representation. 
2 5 [CLAIM 8] The apparatus according to Claim 5, wherein said editing control 

means provides a reproduction image representation of said material, and displays said 
determined speed range avilable for reproduction in the reproduction image 
representation. 
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[CLAIM 9] The apparatus according to Claim 8, wherein said control means 
proveides a indication for indicating the reproduction speed and the display of said speed 
range available for reproduction includes said indication. 

[CLAIM 10] The apparatus according to Claim 5, wherein said editing control 
means provids a reproduction time order representation allocating said items of the 
material data along a time axis to create a content, and displays a representation width of 
said material data being altered according to reproduction time calculated on the basis of 
the reproduced speed. 

[CLAIM 11] The apparatus according to Claim 5, wherein said editing control 
means provides a reproduction image representation of said material data and a 
reproduction time order representation to allocate the material data along a time axis to 
create the contents, proveides a indication for indicating the reproduction speed, the 
display of said speed range available for reproduction includes said indication, and 
displays a representation width of said material data being altered according to 
reproduction time calculated on the basis of the reproduced speed; 

wherein when said representation width of said material data in said reproduction 
time order representation is varied, said indication is altered according to the operation; 
and when said indication is varied, said representation width of said material is altered 
according to the operation. 

[CLAIM 12] The apparatus according to Claim 5, wherein, when said material 
data is set to be reproduced at a reproduction speed within said speed range available for 
reproduction, said editing control means controls said edit-transmitting signal generation 
measns to transmit edit-transmitting signal based on the material data reproduced at said 
set reproduction speed to reproduce the material data at said set reproduction speed. 

[CLAIM 13] A method for controlling an editing image display comprising the 

steps of: 

capturing material data; and 

editting the material data captured and displaying image based on the material 
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data on the deting process; 

wherein, in said editted image items of said material data that is combined with 
the frame rate information as the associated information and not combined with the frame 
rate information as the associated information with said items of the material data being 
distinguished from each other. 

[CLAIM 14] The method according to Claim 13, wherein, a material 
administration representation for displaying a list of items of the material data is provided, 
and said items of the material data that is combined with the associated information and 
not combined with the associated information with said items of the material data being 
distinguished from each other in the material administration is displayed in said material 
administration representation. 

[CLAIM 15] The method according to Claim 13, wherein, a reproduction order 
representation to arrange in a reproduction order to create the contents is provided, and 
said items of the material data that is combined with the as the associated information and 
not combined with the the associated information with said items of the material data 
being distinguished from each other in the reproduction order display is displayed in said 
reproduction order representation . 

[CLAIM 16] The method according to Claim 13, wherein, a reproduction time 
order representation to allocate the material data along a time axis to create the contents is 
provided, and said items of the material data that is combined with the associated 
information and not combined with the the associated information with said items of the 
material data being distinguished from each other in the reproduction order display is 
displayed in said reproduction time order representation. 

[CLAIM 17] The method according to Claim 13, wherein, a speed range 
available for reproduction is set on said material data based on said associated information, 
and said determined display speed range available for reproduction is displayed. 

[CLAIM 18] The method according to Claim 17, wherein, a material 
administration representation for displaying a list of items of the material data, and said 
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determined speed range avilable for reproduction is displayed in said material 
administration representation. 

[CLAIM 19] The method according to Claim 17, wherein, a reproduction order 
representation to arrange in a reproduction order to create the contents is provided, and 
said determined speed range avilable for reproduction is displayed in said reproduction 
order representation. 

[CLAIM 20] The method according to Claim 17, wherein, a reproduction image 
representation of said material is provided, and said determined speed range avilable for 
reproduction is displayed in said reproduction image representation. 

[CLAIM 21] The method according to Claim 20, wherein the display of said 
speed range available for reproduction includes an indication for indicating the 
reproduction speed. 

[CLAIM 22] The method according to Claim 17, wherein, a reproduction time 
order representation to allocate the material data along a time axis to create the contents is 
provided and a representation width of said material data being altered according to 
reproduction time calculated on the basis of the reproduced speed in said reproduction 
time order representation. 

[CLAIM 23] The method according to Claim 17, wherein a reproduction image 
representation of said material data and a reproduction time order representation to 
allocate the material data along a time axis to create the contents are provided, a indication 
for indicating the reproduction speed is provided, the display of said speed range available 
for reproduction includes said indication, and a representation width of said material data 
being altered according to reproduction time calculated on the basis of the reproduced 
speed is displayed; 

wherein, when said representation width of said material data in said reproduction 
time order representation is varied, said indication is altered according to the operation, 
and when said indication is varied, said representation width of said material is altered 
according to the operation. 
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[CLAIM 24] The method according to Claim 17, wherein, when a reproduction 
operation is set to reproduce said material data at a reproduction speed within said speed 
range available for reproduction of said material data, signal based on the material data 
reproducted at said set reproduction speed is transmitted to reproduce the material data at 
5 said set reproduction speed. 

[CLAIM 25] A program for allowing a computer to carry out a method for 
controlling an editing image display, said method comprising the steps of: 

setting display speed available for reproduction based on frame information when 
the frame information can be gotten on editing material data; 
1 0 controlling display according to said frame information; 

displaying indicated reproduction speed when the reproduction speed is indicated; 
calculating reproduction time reproducted said material data at indicated 
reproduction speed on indicated reproduction speed; 

controlling display according to said reproduction time; 
15 displaying reproduction image of said material data at said indicated reproduction 

speed. 

[DETAILED DESCRIPTION OF THE INVENTION] ' 
[0001] 

[TECHNICAL FIELD TO WHICH THE INVENTION BELONGS] 
2 0 The invention relates to an apparatus for controlling contents and a method and 

program for controlling an editing image display of contents. Particularly the editing 
process is performed using determined material data. In the editing prcess a image is 
displayed on based the determined material data or a material data on the editing process. 
In the edited image the items of determined material data that is combined with the frame 
2 5 rate information as the associated information and not combined with the frame rate 

information as the associated information, respectively, is displayed with the items of the 
determined material data being distinguished from each other. 
[0002] 
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[PRIOR ART] 

In generation of contents on audio and/or video for broadcasting, a distribution 
via a communication network or circulation by recording medium, and the like, the 
contents altering motion speed of a subject partially for only a set period has been used in 
5 addition to the contents generated at a reference frame rate in order to obtain enhanced 

effects the creator wishes to create. 

[0003] 

In such the generation of contents altering the motion speed of the subject, a 
higher frame rate relevant to a reference frame rate is illustratively set to generate the 
1 0 contents. Alternatively, a lower frame rate relevant to a reference frame rate is set to 

generate the contents, and then the contents are reproduced at the reference frame rate so 
that quick motion contents can be produced. Further, adapting the frame rate to be set and 
the frame rate at the reproduction allows the motion speed of the subject to be freely 
altered. 
15 [0004] 

Thus, the creator performs an editing process to create the contents using not only 
the contents generated at the reference frame rate but also the contents of altered motion 
speed of the subject, in order to obtain the enhanced effects the creator wishes to create. 

[0005] 

2 0 It has been proposed in Japanese Patent Publication No. Tokkaihei 11-1 77903 that 

the video camera allows time axis to be compressed or extended in order to generate the 
contents altering the frame rate. 
[0006] 

[Patent Document] 

25 Japanese Patent Application Laid-openNo. HEI 11-177930 

[0007] 

[PROBLEMS TO BE SOLVED BY THE INVENTION] 
When the creator performs the editing process using not only the contents 
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generated at the reference frame rate but also the contents of altered motion speed of the 
subject as material, it is not easily determined which of frame rates each of the contents 
has been generated if merely the images of contents are displayed. Further, when it 
performs the speed change on the contents of altered motion speed of the subject, it is 
impossible to determine whether or not the speed change can be performed without any 
deterioration in image quality of the displayed image. If a frame rate is set (hereinafter, 
referred to as, "set frame rate") to 10 times the reference frame rate, the reproduced 
contents are displayed one-tenth in a motion of subject if the contents generated at the set 
frame rate are reproduced at the reference frame rate. Here, it is conceivable that since the 
set frame rate is 10 times the reference frame rate when the creator wants to display the 
contents one-fifth in a motion of subject, the speed change can be easily performed 
without any deterioration in image quality of the displayed image if performing a 
frame-skipping for each frame. It, however, is impossible to determine whether or not the 
speed change can be easily performed without any deterioration in image quality of the 
displayed image, based on merely the displayed image. 
[0008] 

It is an object of the present invention to present a method and an apparatus for 
controlling an editing image display wherein the editing process is easily allowed using 
material generated at the reference frame rate but also material altering the frame rate. 

[0009] 

[MEANS FOR SOLVING THE PROBLEMS] 
According to the present invention, an apparatus for controlling an editing image 
display comprising: 

a material capture means for capturing material data; 

a edit processing means for recording the material data captured by said material 
capture means and editing the recorded material data; 

a edit-transmitting signal generation measns for generating a transmitting signal 
based on the material data by said edit processing means or the material data on the 



Application No.2002-332650 
editing process and transmitting; 

a user interface unit for generating operation signal corresponding to operations; 

and 

a editing control means for controling working of said material capture means, 
said edit processing means, and said edit-transmitting signal generation means based on 
the operation signal, 

wherein said editing control means controls said edit-transmitting signal 
generation means, and transmits a transmitting signal to display items of said material data 
that is combined with the frame rate information as the associated information and not 
combined with the frame rate information as the associated information with said items of 
the material data being distinguished from each other. 

[00010] 

A method for controlling an editing image display comprising the steps of: 
capturing material data; and 

editting the material data captured and displaying image based on the material 
data on the deting process; 

wherein, in said editted image items of said material data that is combined with 
the frame rate information as the associated information and not combined with the frame 
rate information as the associated information with said items of the material data being 
distinguished from each other. 

[00011] 

A program for allowing a computer to carry out a method for controlling an 
editing image display, said method comprising the steps of: 

setting display speed available for reproduction based on frame information when 
the frame information can be gotten on editing material data; 

controlling display according to said frame information; 

displaying indicated reproduction speed when the reproduction speed is indicated; 
calculating reproduction time reproducted said material data at indicated 
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reproduction speed on indicated reproduction speed; 

controlling display according to said reproduction time; 

displaying reproduction image of said material data at said indicated reproduction 

speed. 

[0012] 

In this invention, material data is captured, the editing process is performed using 
the determined material data. In the editing prcess a image is displayed on based the 
determined material data or a material data on the editing process. In the edited image, the 
items of determined material data that is combined with the frame rate information as the 
associated information and not combined with the frame rate information as the associated 
information, respectively, is displayed with the items of the determined material data being 
distinguished from each other. In the edited image, the material control display that shows 
a list of the material data is set up. In the material control display the items of determined 
material data that is combined with the frame rate information as the associated 
information and not combined with the frame rate information as the associated 
information is displayed with the items of the determined material data being 
distinguished from each other. And a reproduction order representation to arrange in a 
reproduction order to create the contents is provided, and the items of the material data 
that is combined with the as the associated information and not combined with the the 
associated information with the items of the material data being distinguished from each 
other in the reproduction order display is displayed in the reproduction order 
representation. Further a reproduction time order representation to allocate the material 
data along a time axis to create the contents is provided, and the items of the material data 
that is combined with the associated information and not combined with the the associated 
information with the items of the material data being distinguished from each other in the 
reproduction order display is displayed in the reproduction time order representation. 
Further a speed range available for reproduction is set on the material data based on the 
associated information, and the determined display speed range available for reproduction 
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is displayed. This speed range available for reproduction is displayed in the material 
administration representation for displaying a list of items of the material data, the 
reproduction order representation to arrange in a reproduction order to create the contents, 
and the reproduction image representation of the material data. The display of the speed 
5 range available for reproduction includes an indication for indicating the reproduction 

speed. The reproduction time order representation to allocate the material data along a 
time axis to create the contents is provided and a representation width of said material data 
being altered according to reproduction time calculated on the basis of the reproduced 
speed in the reproduction time order representation. Further when the representation width 
10 of the material data in the reproduction time order representation is varied, the indication 

is altered according to the operation, and when the indication is varied, the representation 
width of the material is altered according to the operation. 
[0013] 

[EMBODIMENT MODE OF THE INVENTION] 
1 5 Referring to the drawings, the invention will how be described in detail with 

reference to embodiments according to the invention. Fig. 1 shows an entire configuration 
of contents-editing system for editing the contents such as the contents for audio and/or 
video. An imaging apparatus 10 generates image data having its frame rate variably set 
and combines associated information including frame rate information on the frame rate of 
2 0 the image data with this image data to supply an editing apparatus 30 with the combined 

ones as material data DTm. When the imaging apparatus 10 is provided with an audio 
input apparatus 20, it generates audio data to supply the editing apparatus 30 with the 
audio data and the associated information as material data DTm. The material data DTm 
may be also supplied from an apparatus, not shown, other than the imaging apparatus 10. 
2 5 The combination is enough if the image data and the associated information including the 

frame rate information are linked with each other. Thus, if they are recorded on different 
recording media or supplied through different transmission lines, a connection between 
them may be established latter when they includes a frame number and the like 
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corresponding to the association information. In this embodiment, the combination 
includes such the case. 
[0014] 

The editing apparatus 30 receives the material data DTm from the imaging 
5 apparatus 10 and performs editing process on the received material data DTm to produce 

and transmit contents-data DC. The editing apparatus 30 also produces image signal Svm 
concerning the editing and supplies an editing-image-displaying apparatus 40 with it. This 
allows a user to verify editing processes and results of image and the like through 
displayed images on the editing-image-displaying apparatus 40. Similarly, the editing 
1 0 apparatus 30 produces audio signal Sam concerning the editing and supplies an 

editing-audio-transmitting apparatus 41 with it. This allows a user to verify editing 
processes and results of audio and the like through transmitted audio from the 
editing-audio-transmitting apparatus 41. 
[0015] 

15 Fig. 2 is a block diagram showing a configuration of the imaging apparatus 10. 

Imaging unit 12 receives incident light through imaging lens system 11 and an image 
pickup device such as charge coupled device (CCD) mounted within the imaging unit 12 
captures subject image on its image surface. The image pickup device generates imaged 
charge of the subject image though photoelectric transfer characteristics thereof. The 

2 0 imaging unit 12 also reads out the imaged charge thus generated on the basis of driving 

signal CR from a timing generator 142, which will be described later, and generates 
imaged signal Sp having a frame rate that corresponds to the driving signal CR to supply 
camera processing circuit 131 in signal processor 13 with the imaged signal Sp. 
[0016] 

2 5 Based on timing signal CT received from the timing generator 142, the camera 

processing circuit 131 performs various signal processing at timings synchronized with the 
imaged signal Sp. Illustratively, such the signal processing represents noise-reduction 
processing for reducing noise components from the imaged signal Sp by means of 
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correlated dual sampling or the like, transformation processing from the imaged signal Sp 
thus noise-reduced to digital image data, clamp processing of the image data, processing 
on shading correction and pilling-up of deficiency in the image pickup device, y correction, 
contour enhancement processing, knee correction, and the like. Further, the camera 
processing circuit 131 performs other various signal processing under the process 
conditions based on the operation control signal CS received from imaging control circuit 
141 of controller 14. Thus, the camera processing circuit 131 generates image data DV 
according to various signal processing and supplies transmitting unit 15 with the image 
data DV. 

[0017] 

The timing generator 142 of controller 14 generates driving signal CR 
corresponding to operation control signal CS from the imaging control circuit 141 and 
supplies the imaging unit 12 with the driving signal CR, thereby allowing a reading-out 
frequency of imaged charge in the imaging unit 12 to be altered. It, then, also controls a 
frame rate of the imaged signal Sp to be set to a set frame rate FRs based on an operation 
signal PSa from user interface unit 16. For example, frame frequency, 59.94Hz or 29.97Hz 
in the case of NTSC system or frame frequency, 50Hz or 25Hz in the case of PAL system 
is set to the frame frequency of reference frame rate FRr and an operation such that the set 
frame rate FRs is set to the n times the reference frame rate FRr is carried out. If so, a 
frame rate of the imaged signal Sp is controlled to be equal to the n times the reference 
frame rate FRr. 

[0018] 

The timing generator 142 also generates the timing signal CT synchronized with 
the driving signal CR and supplies the camera processing circuit 131 and audio processing 
circuit 132 with the timing signal CT. The timing generator 142 further generates 
associated information DM including the set frame rate FRs that is frame rate of image 
data DV and supplies the transmitting unit 15 with the frame rate information DM. 

[0019] 
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The imaging control circuit 141 of the controller 14 is connected with the user 
interface unit 16. When the imaging apparatus 10 performs switching operation and 
alteration operation of the frame rate, the user interface unit 16 generates operation signal 
PSa corresponding to these operations and supplies the imaging control circuit 141 with 
5 the operation control signal PS. When receiving the operation signal PSa from external 

equipment such as a remote controller, not shown, the user interface unit 16 also supplies 
the imaging control circuit 141 with the operation signal PSa. 
[0020] 

The imaging control circuit 141 generates operation control signal CS to allow 
1 0 the imaging apparatus 10 to be operated according to the operation signal PSa based on 

the operation signal PSa received from the user interface unit 16. It also supplies the 
camera processing circuit 131 and the timing generator 142 with the operation control 
signal CS. 

[0021] 

15 The audio processing circuit 132 receives analog audio signal Sin from the audio 

input apparatus 20. The audio processing circuit 132 also performs sampling process on 
the analog audio signal Sin based on timing signal CT received from the timing generator 
142 to generate digital audio data DA and supply the transmitting unit 15 with the digital 
audio data DA. 

2 0 [0022] 

The transmitting unit 15 combines the image data DV and the audio data DA with 
the associated information DM to generate material data DTm. The transmitting unit 1 5 
supplies the editing apparatus 30 with the material data DTm. When the material data 
DTm or a recording signal produced on the basis of the material data DTm is recorded on 

2 5 recording medium, the material data DTm may be supplied to the editing apparatus 30 by 

reproducing the material data DTm itself or recorded on the recording medium in the 
editing apparatus 30. 
[0023] 
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It is conceivable that as a way to combine the image data DV and the audio data 
DA with the associated information DM, the associated information DM is inserted into a 
data stream of image or a header of the data stream when the image data DV and the audio 
data DA are compressed to generate the material data DTm as the data stream. 
5 [0024] 

Further, it is conceivable that when SDI format standardized as SMPTE (Society 
of Motion Picture and Television Engineers) 259M, Television- 10-Bit 4:2:2 Component 
and 4fsc Composite Digital Signals-Serial Digital Interface, in order to transfer 
non-compressed image and audio data, SDTI format standardized as SMPTE 305M, 
1 0 Television-Serial Data Transport Interface (SDTI), in order to transfer compressed image 

and audio data, or SDTI-CP format standardized as SMPTE 326M, Television-SDTI 
Content Package Format (SDTI-CP) that obtained by further defining SDTI format is 
utilized, the associated information DM may be inserted into a signal having each format 
as UMID data standardized as SMPTE 330M, Television-Unique Material Identifier 
15 (UMID). 

[0025] 

The above imaging apparatus 10 alters read-out frequency of imaged charge in 
the imaging unit 12 to generate the imaged signal Sp of a predetermined set frame rate 
FRs. If, however, the imaging apparatus 10 alters no read-out frequency of imaged charge 

2 0 in the imaging unit 12, the imaged signal Sp of the set frame rate FRs may be generated. 

This is, generating image data DVa having a constant frame rate higher than the set frame 
rate FRs and extracting image data of only the set frame rate FRs from the image data DVa 
allows the image data DVa of a predetermined set frame rate FRs to be generated. Fig. 3 
shows a configuration of such a case. In Fig. 3, like reference numbers refer to like 

2 5 elements shown in Fig. 2 and thus, the detailed description thereof is omitted. 

[0026] 

Timing generator 182 in controller 18 generates driving signal CRa 
corresponding to the highest value of the set frame rate FRs that is set through user 
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interface unit 16 and supplies the imaging unit 12 with the driving signal CRa. Based on 
the driving signal CRa, the imaging unit 12 generates imaged signal Spa having a fixed 
frame rate FRq higher than a reference frame rate. The imaging unit 12 then supplies 
camera-processing circuit 131 with the imaged signal Spa. When the set frame rate FRs 
5 may be altered up to n times the reference frame rate FRr, the imaging unit 12 generates 

the imaged signal Spa having a frame rate n times the reference frame rate FRr, and 
supplies the camera processing circuit 131 with the imaged signal Spa. 
[0027] 

The timing generator 182 also generates timing signal CTa synchronized with the 

1 o driving signal CRa and supplies the camera processing circuit 1 3 1 and audio processing 

circuit 132 in signal processor 17 and effective frame signal generation circuit 183 in the 
controller 18 with the timing signal CTa. 
[0028] 

The camera processing circuit 131 generates image data DVa of fixed frame rate 
1 5 FRq based on the imaged signal Spa and supplies the effective data selection circuit 1 7 1 

with the image data DVa. The audio signal processing circuit 132 carries out a sampling 
based on the timing signal CTa of a fixed frequency to generate audio data DAa and 
supplies the effective data selection circuit 171 with the audio data DAa. 
[0029] 

2 o Imaging control circuit 181 generates a set information signal CF indicating a set 

frame rate FRs based on operational signal PSa received from the user interface unit 16 
and supplies the effective frame signal generation circuit 1 83 with the set information 
signal CF. 

[0030] 

2 5 The effective frame signal generation circuit 1 83 extracts data on a frame basis 

from the image data DVa based on a ratio of the fixed value of frame rate FRq of the 
image data DVa to a value of the set frame rate FRs indicated in the set information signal 
CF and generates extraction control signal CC for generating image data DV of the set 
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frame rate FRs. The effective frame signal generation circuit 183 also synchronizes the 
extraction signal CC with the timing signal CTa and supplies the effective data selection 
circuit 171 with the synchronized extraction control signal CC. When the frame rate FRq 
of the image data DVa is n times the reference frame rate FRr and the set frame rate FRs is 
n/2 times the reference frame rate FRr, the effective frame signal generation circuit 183 
generates the extraction control signal CC for controlling data extraction on a frame basis 
every other frame from the image data DVa and supplies the effective data selection circuit 
171 with the extraction control signal CC synchronized with the timing signal CTa. 
The effective frame signal generation circuit 183 further generates associated information 
DM having the set frame rate FRs as the frame rate information based on the set 
information signal CF and supplies the transmitting unit 15 with the associated 
information DM. 
[0031] 

The effective data selection circuit 171 extracts the image data DVa and the audio 
data DAa of the frame indicated by the extraction control signal CC and supplies the 
transmitting unit 15 with them as the image data DV and the audio data DA. It is 
conceivable that the effective frame signal generation circuit 183 may supply the effective 
data selection circuit 171 with the associated information DM including the set frame rate 
FRs as the frame rate information, and the effective data selection circuit 171 may perform 
frame-skipping on the audio data DAa according to a ratio of the set frame rate FRs to a 
frame rate when the audio data DAa is generated. When the frame rate FRq in generating 
the audio data DAa is n times the reference frame rate FRr and the set frame rate FRs is 
n/2 times the reference frame rate FRr, the effective data selection circuit 171 performs the 
frame-skipping on the audio data DAa every other sample. In this case, since an interval of 
the frame-skipping may be shortened as compared by a case where performing the 
frame-skipping on the audio data DAa on a frame basis, audio having an excellent sound 
quality may be get based on the audio data DA. 
[0032] 
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Thus, the image data DVa having a fixed frame frequency makes unnecessary the 
alteration of operation frequencies in the imaging unit 12 and the camera processing 
circuit 131 of the signal processor 17. This allows configurations of the imaging unit 12 
and the camera processing circuit 131 to be made simpler. Since only the data extraction 
on a frame basis from the image data DVa may generate image data DV of the set frame 
rate FRs, generating the image data DV of a predetermined set frame rate FRs from the 
image data Dva is easily allowed. 

[0033] 

When the imaging apparatus is provided with video memory or an adder and a 
divider, it may generate the image data DV by adding the image data every the 
predetermined frames. This allows a variable range of frame rate in the imaged signal Sp 
to be limited. Adding the imaged signal Sp of n frames and dividing the signal level by n 
allows the signal having a frame rate 1/n times the imaged signal Sp to be obtained even if 
a frame rate of the imaged signal Sp is not divided by n. 

[0034] 

Figs. 4 and 5 are diagrams showing relationship between the image data DV 
generated by the imaging apparatus 10, 10a and the associated information DM. When the 
set frame rate FRs is set to the one equal to or twice the reference frame rate FRr as shown 
in Fig. 4A, the associated information DM that includes the frame rate information 
DM-FRs indicating the set frame rate FRs, as shown in Fig. 4C, is combined with image 
data DV as shown in Fig. 4B (in Fig., frame images based on the image data DV are 
shown). Fig. 4D illustrates a relationship between time and the frame images. The frame 
rate information DM-FRs may include a magnification of the set frame rate FRs to the 
reference frame rate FRr in addition to the set frame rate FRs. The frame rate information 
DM-FRs is indicated by the magnification in Figs. 4C and Figs below. 

[0035] 

When the set frame rate FRs is set to the one equal to or half as much as the 
reference frame rate FRr as shown in Fig. 5A, the associated information DM that 
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includes the frame rate information DM-FRs indicating the set frame rate FRs, as shown 
in Fig. 5C is combined with image data DV as shown in Fig. 5B (in Fig. 5B, frame images 
based on the image data DV are shown). Fig. 5D illustrates a relationship between time 
and the frame images. 
[0036] 

The following will be described on the editing apparatus 30. Fig. 6 shows a 
configuration of the editing apparatus 30. In the editing apparatus 30 receiving material 
data DTm, an information detection circuit 3 1 1 in a material capture unit 3 1 receives the 
material data DTm. The information detection circuit 3 1 1 detects associated information 
DM from the material data DTm. The information detection circuit 3 1 1 supplies a 
database-processing circuit 312 with the associated information DM thus detected. The 
database-processing circuit 312 also receives image data DV and audio data DA included 
in the material data DTm. 

[0037] 

The database-processing circuit 312 records the image data DV, the audio data 
DA, and the associated information DM detected in the information detection circuit 311 
on data recording apparatus 321 in edition processing unit 32 with the image data DV and 
the audio data DA being linked to the associated information DM. The 
database-processing circuit 312 generates database information DB that allows contents of 
material data to be easily verified on the basis of the associated information DM and the 
image data DV and the audio data DA linked to the associated information DM, which 
have been recorded on the data recording apparatus 321. The database-processing circuit 
312 then supplies editing control unit 33 with the generated database information DB. The 
database information DB consists of pieces of information such as information for 
allowing contents of material data to be recognized (for example, thumbnails), a time 
length of the material data, set frame rate FRs, and recorded locations on the data 
recording apparatus 321. 

[0038] 
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The editing control unit 33 generates image data DVg for allowing editing 
process to be carried out under graphical user interface (GUI) environment and image data 
DVi for allowing representation of contents of material data. The editing control unit 33 
then supplies image-transmitting signal generation circuit 351 with the image data DVg 
and DVi. The image-transmitting signal generation circuit 351 generates image signal Svm 
based on the image data DVg, DVi thus supplied and transmits it to the 
editing-image-displaying apparatus 40. Thus, supplying the editing-image-displaying 
apparatus 40 with the image signal Svm allows information on what kind of the material 
data is recorded or the like to be displayed on a screen of the editing-image-displaying 
apparatus 40. This image signal Svm is used for image representation for editing. 

[0039] 

The editing control unit 33 also controls post-production process. User interface 
unit 34 that is connected with the editing control unit 33 supplies operational signal PSe to 
the editing control unit 33 using representation under GUI environment. When the 
operational signal PSe indicates the selection of any kind of the material data, the editing 
control unit 33 generates reading control signal RC according to the operational signal PSe. 
The editing control unit 33 then supplies write/read processing circuit 322 in the edition 
processing unit 32 with the reading control signal RC. When the operational signal PSe 
relates to editing operation such as processing and linkage of the material data thus read, 
the editing control unit 33 generates editing control signal ET according to the operational 
signal PSe. The editing control unit 33 then supplies signal editing circuit 323 in the 
edition processing unit 32 with the editing control signal ET. If editing of the material data 
is finished to bring contents-data to completion, the editing control unit 33 generates 
writing control signal WC according to the operational signal PSe when the operational 
signal PSe indicates recording operation of the contents-data on the data recording 
apparatus 321. The editing control unit 33 then supplies write/read processing circuit 322 
with the writing control signal WC. The editing control unit 33 also generates transmitting 
control signal RP according to the operational signal PSe when the operational signal PSe 
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indicates data transmission of the contents-data. The editing control unit 33 then supplies 
the write/read processing circuit 322 with the transmitting control signal RP. The editing 
control unit 33 also generates speed range setting signal LP according to the operational 
signal PSe when the operational signal PSe specifies reproduction speed range of the 
5 contents-data. The editing control unit 33 then supplies the signal editing circuit 323 with 

the speed range setting signal LP. 
[0040] 

Based on the reading control signal RC, the write/read processing circuit 322 
reads required material data out of the data recording apparatus 321 to supply the signal 

1 0 editing circuit 323 with it. Based on the writing control signal WC, the write/read 

processing circuit 322 also writes the complete contents-data on the data recording 
apparatus 321. Based on the transmitting control signal RP, the write/read processing 
circuit 322 reads required contents-data DC out of the data recording apparatus 321 to 
transmit it. 

15 [0041] 

The signal editing circuit 323 performs editing process such as processing, 
linkage, and deletion of image and audio data based on the editing control signal ET using 
the image data DV and the audio data DA included in the material data read out of the data 
recording apparatus 321. In this case, the signal editing circuit 323 supplies the 

2 0 image-transmitting signal generation circuit 351 with the image data DVe before, during 

or after the editing thereof. The signal editing circuit 323 also supplies the 
audio-transmitting signal generation circuit 352 with the audio data DAe before, during or 
after the editing thereof. When frame rates of image and audio data are altered according 
to the editing process, the signal editing circuit 323 alters the associated information DM 

25 together with the edited image and audio data. The signal editing circuit 323 further 

combines the associated information DMc corresponding to the edited image data DV and 
the edited audio data DA with the edited image data DV and the edited audio data DA to 
generate the contents-data DC. When the signal editing circuit 323 receives the speed 
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range setting signal LP, the signal editing circuit 323 also combines speed range 
information on the reproduction speed range of the contents-data DC as the associated 
information DMc based on the speed range setting signal LP. When the signal editing 
circuit 323 receives information on title and recommended reproduction speed of the 
5 contents from the user interface unit 34, the signal editing circuit 323 also combines the 

information as the associated information DMc. When the signal editing circuit 323 
receives information on a reproduction time length of the contents-data according to the 
editing process, the signal editing circuit 323 may combines this information as the 
associated information DMc. When the signal editing circuit 323 receives information on 
1 0 the highest reproduction speed available for reproduction of the contents-data, the signal 

editing circuit 323 may combines the information on the highest reproduction speed as the 
associated information DMc. 
[0042] 

The image -transmitting signal generation circuit 351 of edited transmitting signal 
1 5 generation unit 3 5 generates image signal Svm based on the image data DVg, DVi 

supplied through the editing control unit 33 and transmits it to the 

editing-image-displaying apparatus 40, as described above. This allows information on the 
material data to be displayed under GUI environment. Generating the image signal Svm 
based on the image data DVe received from the signal editing circuit 323 allows image 
2 0 before, during or after the editing thereof to be verified on the screen of the 

editing-image-displaying apparatus 40. 
[0043] 

The audio-transmitting signal generation circuit 352 transfers the audio data DAe 
received from the signal editing circuit 323 to analog audio signal Sam and brings it to a 
25 desired signal level to supply the editing-audio-transmitting apparatus 41 composed of a 

speaker, a head hone, and the like with it. This allows audio before, during or after the 
editing thereof to be verified through audio transmitted from the 
editing-audio-transmitting apparatus 41. 
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[0044] 

Thus, when the contents-data Dc is complete in the editing apparatus 30 
according to the post-production process using the material data DTm, the editing 
apparatus 30 transmits the complete contents-data DC. 
5 [0045] 

The following will be described on operations of the editing apparatus 30. Fig. 7 
illustrates GUI representation displayed on the editing-image-displaying apparatus 40 for 
an editing operation. In the GUI representation, a material administration browser window 
401 for displaying a list of items of the material data as material administration display is 

1 0 provided at an upper-left side thereof; a story board window 402 for displaying items of 

the material data arranged in a reproduction order to create the contents as reproduction 
order display is provided at an lower-left side thereof; a monitor viewer window 403 for 
displaying an image before or after the editing process as reproduced image display is 
provided at an upper-right side thereof; and a time line window 404 for allocating the 

1 5 material data along a time axis to create the contents as reproduction time order display is 

provided at a lower-left side thereof. An operation control portion 405 is provided between 
the monitor viewer window 403 and the time line window 404. Locations and sharps of 
the material administration browser window 401, the story board window 402, the monitor 
viewer window 403, the time line window 404, and the operation control portion 405 are 

2 0 respectively illustrative ones and thus, of course, they are not limited to the above 

locations and sharps. 
[0046] 

The material administration browser window 401 shows a list of items of the 
material data stored in the data recording apparatus 321. For each item of the material data, 
25 a stamp view (a thumbnail view) for indicating a title, a length, contents of items of the 

stored material data, a speed range available for reproduction wherein noise-less natural 
image may be reproduced, and the like are shown. 

[0047] 
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The story board window 402 serves as a working area for a creation of contents. 
Arranging the items of material data on the reproduced order allows the contents to be 
created. The story board window 402 also display a speed range available for 
reproduction. 

The monitor viewer window 403 represents not only an image based on the material data 
but also a variable speed bar for indicating a speed range available for reproduction and a 
location of reproduction speed within the speed range. 
[0048] 

The time line window 404 serves as a working area for a production of contents 
according to a more detailed method to allocate items of the material data along a time 
axis. The time line window 404 also alters an indicated width of the material data 
according to a reproduction time calculated based on the production speed. The operation 
control portion 405 represents an operation key for reproducing the items of material data 
and the contents arranged on the time line window 404. 

[0049] 

Based on the represented GUI images and the operational signal PSe received 
from the user interface unit 34, the editing control unit 33 determines whether or not a 
drag-and-drop operation, a key-in operation in the operation control portion 405, and the 
like are carried out. If the editing control unit 33 determines that the drag-and-drop 
operation and the like have been carried out, it controls operations of the editing apparatus 
30 according to such the drag-and-drop operation and the like. 

[0050] 

Fig. 8 is a flowchart showing a display control at the editing operation. At step 
ST11, the editing apparatus 30 determines whether or not the processing of the material 
data is requested. If no processing of the material data is requested, the process goes back 
to the step ST1 1 . If the processing of the material data is requested, the process goes to 
step ST12. 

[0051] 
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At step ST12, the editing apparatus 30 determines whether or not frame rate 
information DM-FRs indicating the set frame rate FRs is obtained from the material data 
to be edited. When the editing apparatus 30 determines that no frame rate information 
DM-FRs indicating the set frame rate FRs is obtained from the material data referring to 
5 the database information DB, the process goes to the step ST1 3 where the editing 

apparatus 30 performs display controls so that the material data has same reproduction 
speed as a normal reproduction speed. When the editing apparatus 30 determines that the 
frame rate information DM-FRs is obtained from the material data, the process then goes 
to step ST14. 

10 [0052] 

At step ST 14, a speed available for reproduction is set based on the set frame rate 
FRs indicated by the frame rate information DM-FRs. This speed is set so that a noise-less 
natural reproduction image can be realized through a frame-skipping of a predetermined 
frame period or a frame repetition. If a multiple of the set frame rate FRs to the reference 

1 5 frame rate FRr is more than one, namely, FRs/FRr>l , the speed available for reproduction 

is set by searching for the divisors of this multiple other than one and calculating the 
reciprocals of the searched divisors. If the set frame rate FRs is 10 times the reference 
frame rate FRr, the divisors of this multiple, 10 other than one are of 2, 4, 5, and 10 and 
thus, the reciprocals thereof are of 1/2, 1/4, 1/5, and 1/10, respectively. Selection of one of 

2 0 the reciprocals of divisors allows the speed available for reproduction to be set to less than 

the same reproduction speed as normal reproduction speed. When the speed available for 
reproduction not less than the same reproduction speed as the normal reproduction speed 
is set to positive integer multiples of the normal reproduction speed, the frame-skipping 
periods in each of the speeds available for reproduction may be equal to each other. An 

2 5 upper limit of the speed available for reproduction is set to a speed available for grasping 

the contents of material data easily. 
[0053] 

If a multiple of the set frame rate FRs to the reference frame rate FRr is less than 
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one, namely, FRs/FRr<l, the speed available for reproduction is set by searching for the 
reciprocals of this multiple and calculating the divisors of the searched reciprocals other 
than one or integer multiples of the searched reciprocals. If the set frame rate FRs is 1/6 
times the reference frame rate FRr, the divisors of the searched reciprocals other than one 
5 are of 2, 3, and 6 and the integer multiples of the reciprocals are of 6, 12, 18, — , 

respectively. Selection of one of them allows the speed available for reproduction to be set. 
This allows frame-skipping periods or numbers of the frame repetitions in each of the 
speeds available for reproduction may be equal to each other. The speed available for 
reproduction of less than the same speed as the normal reproduction speed is set to the 
1 0 one/positive multiples the normal reproduction speed. This allows numbers of the frame 

repetitions in each of the speeds available for reproduction to be equal to each other. Since, 
however, this is concerned with a repetition of images, a lower limit of the speed available 
for reproduction may be set to the same reproduction speed as the normal reproduction 
speed. 

15 [0054] 

At step ST15, the editing apparatus 30 performs display control corresponding to 
the frame rate information DM-FRs, namely, to display so that it can be determined that 
the material data is combined with the frame rate information DM-FRs indicating the set 
frame rate FRs, as the associated information, and to display the speed range available for 

2 0 reproduction set based on the set frame rate FRs. For example, in the material 

administration browser window 401 and the story board window 402, in order to easily 
determine that the material data is combined with the frame rate information DM-FRs, a 
contour, a color, and the like of a frame of stamp view 401a, 402a are made different from 
them of the material data that is not combined with the frame rate information DM-FRs. 

2 5 When an information-representing regions 40 lb, 402b each for representing information 

on the material are provided, the speeds available for reproduction set in step ST 14 are 
displayed in each of the regions. For example, in Fig.7 and Figs.9, which will be described 
later, when the material data is combined with the frame rate information DM-FRs, the 
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frame having its enlarged width is displayed and the speed range available for 
reproduction is indicated as being "xO. 1 through x2.0". 
[0055] 

In the time line window 404, similar to the cases of the material administration 
5 browser window 401 and the story board window 402, it is possible to easily determine 

that the material data is combined with the frame rate information DM-FRs indicating the 
set frame rate FRs, as the associated information. For example, a contour, and a color of a 
frame of frame image representation region 404a are made different from them of the 
material data that is not combined with the frame rate information DM-FRs. In the time 

1 0 line window 404, an operation is also carried out such that an indicated width of the frame 

image representation region 404a is adjusted along a horizontal direction, i.e., time axis 
direction thereof according to the set frame rate FRs. If the set frame rate is greater than 
the reference frame rate FRr, a number of frame images per unit of time increases. Thus, 
an indicated width of the frame image representation region 404a is enlarged along the 

1 5 horizontal direction. If the set frame rate is smaller than the reference frame rate FRr, a 

number of frame images per unit of time decreases. Thus, an indicated width of the frame 
image representation region 404a is shortened along the horizontal direction. The monitor 
viewer window 403a is provided with a reproduction speed indication 403b and a console 
indication 403c for variable speed corresponding to the set speeds available for 

2 0 reproduction in addition to material image view 403a based on the material data to display 

the reproduced image, the reproduction speed, and the speeds available for reproduction. 
At the same time, a sliding cursor 403d represented by a short bold line in the console 
indication 403c for variable speed is slid to a cursor position representing the reproduction 
speed. Thus, this allows the control direction to be determined. 

25 [0056] 

At step ST 16, the editing apparatus 30 determines whether or not the 
reproduction speed is indicated via the user interface unit 34. When an operation is carried 
out such that the indicated width of the frame image representation region 404a is altered 



Application No.2002-332650 
or the sliding cursor 403d in the console indication 403c represented in the monitor viewer 
window 403 is slid, the process goes to step ST17. When no reproduction speed is 
indicated, the process goes back to step ST16. 
[0057] 

At step ST17, the editing apparatus 30 determines the indicated reproduction 
speed to display the reproduction speed thus determined. For example, when an operation 
such that the indicated width of the frame image representation region 404a is altered is 
carried out in the time line window 404, a reproduction speed is selected in sequence from 
the speeds available for reproduction set at step ST 14 depending on the operational 
direction to set the reproduction speed when completing the operation. When the sliding 
cursor 403d in the console indication 403c represented in the monitor viewer window 403 
is slid, a reproduction speed is selected from the speeds .available for reproduction set at 
step ST 14 depending on a cursor position to set the selected speed to the reproduction 
speed. 

[0058] 

When the reproduction speed is determined, the indicated reproduction speed is 
shown in the reproduction speed indication 403b of the monitor viewer window 403. 
When a reproduction speed is indicated according to the operation such that the indicated 
width of the frame image representation region 404a is altered with the sliding cursor 
403d in the console indication 403c and the indicated width of the frame image 
representation region 404a being in synchronization with each other, the sliding cursor 
403d in the console indication 403c represented in the monitor viewer window 403 is slid 
to a position corresponding to the indicated reproduction speed to display the reproduction 
speed. When a reproduction speed is indicated according to the operation such that the 
sliding cursor 403d in the console indication 403c represented in the monitor viewer 
window 403 is slid, the indicated width of the frame image representation region 404a is 
altered to a width according to the corresponding reproduction speed. Thus, the sliding 
cursor 403d in the console indication 403c and the indicated width of the frame image 
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representation region 404a being in synchronization with each other allows the 
reproduction speed to be correctly altered using either. of these two methods. 
[0059] 

At step 18, based on the indicated reproduction speed, the editing apparatus 30 
5 calculates reproduction time when reproducing the material data at the indicated 

reproduction speed. For example, when the set frame rate FRs is 10 times the reference 
frame rate FRr, a slow speed available for reproduction is of any one of "1/10, 1/5, 1/4 and 
1/2" of the normal reproduction speed. For example, if the reproduction time of the 
material is 30 seconds when the reproduction speed is the same speed as normal 
1 0 reproduction speed, the reproduction time when the reproduction speed is 1/10 times the 

normal reproduction speed is 300 seconds. The reproduction time when the reproduction 
speed is 1/5 times the normal reproduction speed is 150 seconds. Thus, multiplying the 
reproduction time when the reproduction speed is the same speed as normal reproduction 
speed by the reciprocal of the reproduction speed allows the reproduction time to be 
15 calculated. 

[0060] 

At step ST19, the editing apparatus 30 performs a display control according to the 
reproduction time thus calculated. For example, the indicated width of the frame image 
representation region 404a in the time line window 404 is altered according to the 
2 0 reproduction time calculated at step ST1 8, and the process goes back to step ST1 6. During 

reproduction operation of the material data, the reproduced image when reproducing the 
material data at the indicated reproduction speed is displayed as the monitor viewer 
window 403a for displaying the material image of the monitor viewer window 403. 
[0061] 

25 If the reproduction time on a scene of a part of the material data is altered, the 

scene is split and then, the above process is conducted on the split scenes, thereby 
allowing alteration of the reproduction time. 
[0062] 
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Figs.9 are illustration each for GUI representation at an editing operation using 
the material data combined with the frame rate information DM-FRs indicating that, for 
example, the set frame rate FRs is 10 times the reference frame rate FRr. Fig. 9A 
illustrates the GUI representation where the reproduction speed is the same speed as the 
5 normal reproduction speed. Fig. 9B illustrates the GUI representation where the 

reproduction speed is a tenth the normal reproduction speed. Fig. 9C illustrates the GUI 
representation where the reproduction speed is twice as much as the normal reproduction 
speed. Concerning the material data combined with the information on the set frame rate 
FRs, a frame representation of viewer is carried out in such a way different from a case of 
1 0 the material data not combined with the frame rate information DM-FRs indicating the set 

frame rate FRs in the story board window 402 and the time line window 404. 
[0063] 

When reproducing the selected material data, the set frame rate FRs is determined 
on the basis of the frame rate information DM-FRs, and then, such a calculation that the 

1 5 reproduction speed FP multiplied by the set frame rate FRs equals a determined value FD 

is performed. Reproduction process conditions are determined on the determined value FD. 
When the reproduction speed is the same speed as the normal reproduction speed as 
shown in Fig. 9A and the set frame rate FRs is 10 times the reference frame rate FRr, the 
reproduction speed FP multiplied by the set frame rate FRs equals the determined value 

2 0 FD as (1*10=10). Fig. 1 OA illustrates images based on the image data DV where the set 

frame rate FRs is 10 times the reference frame rate FRr. Fig. 10B illustrates the frame rate 
information DM-FRs indicating the set frame rate FRs of each of the frame images. Fig. 
10C illustrates the absolute frame numbers AN of the frame images. 
[0064] 

25 When the determined value FD is set to 10, namely, FD=10, as shown in Figs. 

10D and 10E, the image signal Svm is generated using every 10 frames, namely, with the 
image data of nine frames being skipped. This allows reproduced image having same 
speed as the normal reproduction speed to be represented on the monitor viewer window 
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403 based on the image signal Svm. Fig. 10D illustrates the absolute frame numbers AN 
of the frame images. Fig. 10E illustrates frame images represented on the monitor viewer 
window 403a of the monitor viewer window 403 by the image signal Svm. 
[0065] 

5 An operation is carried out such that an indicated width of a frame image 

representation region 404a in the time line window 404 is enlarged along a direction 
indicated by an arrow A or a sliding cursor 403d represented by a short bold line in the 
console indication 403c for variable speeds is slid in a direction indicated by an arrow B. 
In this case, a processing as the slow motion of reproduction is performed. For example, 

10 when the speed available for reproduction is set to 1/10 times the normal reproduction 

speed, a reproduction speed indication 403b in the monitor viewer window 403 is altered 
to "x 0. 1" as shown in Fig. 9B. At the same time, the sliding cursor 404d is slid to a cursor 
position representing a tenth times the normal reproduction speed in the console indication 
403c. Since the long reproduction time is required, the indicated width of the frame image 

15 representation region 404a is also enlarged, as shown in Fig. 9B. 

[0066] 

If the speed available for reproduction is set to 1/10 times the normal 
reproduction speed as shown in Fig. 9B, the determined value FD is set to one, namely, 
FD=10*(1/10)=1. When the determined value FD is set to one, namely, FD=1, the image 
2 0 signal Svm is generated using every FD=1 frame of the image data DV, namely, without 

any frame being skipped. This allows the reproduced images having a tenth times the 
normal reproduction speed to be represented on the monitor viewer window 403a of the 
monitor viewer window 403 based on the image signal Svm, as shown in Fig. 10A. 
[0067] 

25 An operation is carried out such that an indicated width of the frame image 

representation region 404a in the time line window 404 is shortened along a direction 
indicated by an arrow B or a sliding cursor 404d in the console indication 403c is slid on a 
direction indicated by an arrow A in the monitor viewer window 403. In this case, a 
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processing as the quick motion of reproduction is performed. For example, when the speed 
available for reproduction is set to twice as much as the normal reproduction speed, a 
reproduction speed indication 403b in the monitor viewer window 403 is altered to "x 2.0" 
as shown in Fig. 9C. At the same time, the sliding cursor 403d is slid to a cursor position 
representing twice of the normal reproduction speed in the console indication 403c. Since 
the short reproduction time is required, the indicated width of the frame image 
representation region 404a is shortened, as shown in Fig. 9C. 
[0068] 

If the speed available for reproduction is set to twice of the normal reproduction 
speed as shown in Fig. 9C, the determined value FD is set to 20, namely, FD=1 0*2=20. 
When the determined value FD is set to 20, namely, FD=20, the image signal Svm is 
generated using every 20 frames, namely, with the image data of nineteen frames being 
skipped. This allows the reproduced images having twice speed as much as the normal 
reproduction speed to be represented on the monitor viewer window 403 based on the 
image signal Svm, as shown in Figs. 10F and 10G. Fig. 10F illustrates the absolute frame 
numbers AN of the frame images. Fig. 10G illustrates frame images represented the 
monitor viewer window 403a of the monitor viewer window 403 by the image signal Svm. 

[0069] 

Thus, the noise-less slow reproduction may be easily realized using the frame rate 
information DM-FRs and frame-skipping on a frame basis based on the reproduction 
speed. An operation for altering reproduction speed may be realized during the 
reproduction process, so the reproduced image may be verified at a desired reproduction 
speed at easy. 

[0070] 

Since the reproduction speed and the indicated width are displayed with them 
being linked with each other, a relationship between the reproduction speed and the 
expands and contracts of time axis may be easily grasped, thereby allowing the editing 
operation to be easily conducted. Further, when the frame image representation region 
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404a is provided with an indication for indicating a direction where the indicated width of 
the frame image representation region 404a can be altered, namely, a direction where the 
reproduction speed can be altered, for example, an arrow indication illustrated by the 
dotted lines in Figs. 9, it is possible to find out the direction where the reproduction speed 
can be altered, at easy. 
[0071] 

Fig. 11 is a flowchart showing an editing process operation using the GUI 
representation for the performing display control, for example, the one shown in Fig. 8, 
and for the editing, for example, the one shown in Fig. 9. At step ST21, the editing 
apparatus 30 captures the material data. It then records the captured material data on the 
data recording apparatus 321 and generates the database information DB. 

[0072] 

At step ST22, the editing apparatus 30 displays the contents of the captured 
material data using the database information DB. Displaying the contents of the captured 
material data is carried out so that the material administration browser window 401 may 
display a stamp view and information of the captured material data. When any material 
data displayed on the material administration browser window 401 is selected, the monitor 
viewer window 403 displays image of the selected material data. When a reproduction 
control key operation is done in the operation control portion 405 provided between the 
monitor viewer window 403 and the time line window 404, the editing apparatus 30 
reproduces the material data, stops the reproduction, and reproduces the material at 
various speeds, according to the key operation and then the monitor viewer window 403 
displays the reproduced image. 

[0073] 

At step ST23, the editing apparatus 30 determines whether or not the material 
data is selected. If the material data is not selected, the process goes back to step ST23. If 
the material data is selected, the process goes to step ST24. When the material data is 
selected so that a drag-and-drop is done from the material administration browser window 
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401 to the time line window 404, the process goes to step ST24. 
[0074] 

At step ST24, the editing apparatus 30 arranges frame images generated based on 
the selected material data in an imaged time order and displays them on the frame image 
representation region 404a of the time line window 404. 

[0075] 

At step ST25, the editing apparatus 30 processes the material data according to 
various editing operations such as cut and paste or replacement of the frame images 
displayed on the time line window 404. The editing apparatus 30 then generates desired 
image and audio data to create the contents-data. Alternatively, the editing apparatus 30 
combines the associated information DMc indicating the set frame rate and the like with 
the image and audio data to create the contents-data. . 

[0076] 

At step ST26, the editing apparatus 30 performs various completion processes on 
the contents-data created at step ST25 such as process on bridges of the material data, 
addition enhancement of effects, and synthesis of image and audio to be added. At step 
ST27, the editing apparatus 30 performs transmitting process on the contents-data. That is, 
the editing apparatus 30 transmits the complete contents-data as program data for 
broadcasting to a program-sending apparatus. Alternatively, it transmits the complete 
contents-data as contents-data for distribution to a contents-sending apparatus. The editing 
apparatus 30 performs authoring process such as addition of information for carrying out a 
special reproduction and the like to the complete contents-data, the special reproduction 
corresponding to a menu list for reproducing the contents or a reproduction menu. The 
editing apparatus 30 then records the data thus authored on the recording medium. 

[0077] 

The above editing process by the editing apparatus 30 may be also realized 
through a software process using a computer. Fig. 12 shows a configuration of a 
contents-editing system for editing contents by means of software. 
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[0078] 

The computer builds in a central processing unit (CPU) 381 as shown in Fig. 12. 
ROM 382, RAM 383, data storage unit 384 composed of hard disk drive with a large 
storage capacity and the like, and an input/output interface 385 are connected with the 
CPU 381 via bus 390. Signal-receiving unit 391, signal-transmitting unit 392, and 
recording medium drive 393 are connected with the input/output interface 385. 

[0079] 

The CPU 381 run programs stored on the ROM 382, the RAM 383 or the data 
storage unit 384 to perform the editing processes as shown in Fig. 8 and Fig. 1 1 . The data 
storage unit 384 stores material data, which the signal-receiving unit 391 receives, via the 
input/output interface 385 and the bus 390. Material data to be used for editing is read 
from the material data stored on the data storage unit 384, and the editing process is 
performed using it to store the complete contents-data on the data storage unit 384 again. 
The complete contents-data stored on the data storage.unit 384 is read out and transmitted 
through the signal-transmitting unit 392. 

[0080] 

It is conceivable that a program for performing the editing process may be 
previously stored on the ROM 382 or the data storage unit 384. A program for performing 
contents-sending process stored on the recording medium or a program stored on the 
recording medium may be read out and run. A program may be transmitted through wired 
or wireless transmission line. A received program may be run in the computer. 

[0081] 

Thus, according to the embodiments, the material data that is combined with the 
frame rate information as the associated information and not combined with the frame rate 
information is displayed, respectively, so that the material data can be determined. This 
allows a scene available for reproducing a slow motion image to be easily found out using 
the frame rate information of the material data. A speed range available for reproduction is 
displayed on the basis of the associated information, thereby allowing a variable speed 
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reproduction to be realized setting a speed within the speed range. This permits effects of 
image expression to be easily enhanced. 
[0082] 

The editing apparatus 30 may also provide the contents having a wide speed 
range available for reproduction using the material data effectively if it may generate the 
contents-data without any frame rate of the material data being reduced. 

[0083] 

[EFFECT OF THE INVENTION] 

According to the invention, a material data is captured, editing process is carried 
out using the material data captured, in the editing process image is displayed based on the 
material data captured or the material data on the editing process, in the editing image the 
material data that is combined with the frame rate information as the associated 
information and not combined with the frame rate information as the associated 
information with said items of the material data being distinguished from each other is 
displayed. Thus it is easy to find scene that can be reproduce slowly using the frame rate 
information on the material data. 

[0084] 

A speed range available for reproduction is set on the material data based on the 
associated information, and the determined display speed range available for reproduction 
is displayed. Thus it is easy to set the reproduction that can be reproduced without noise. 
And. a reproduction time order representation to allocate the material data along a time 
axis to create contents is provided and a representation width of the material data being 
altered according to reproduction time calculated on the basis of the reproduced speed in 
said reproduction time order representation. Thus expansion and contraction of time axis 
can be catched visually. Further, when the representation width of the material data in the 
reproduction time order representation is varied, the indication is altered according to the 
operation; and when the indication is varied, the representation width of said material is 
altered according to the operation. Thus the relation between the reproduction speed and 
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the expansion and contraction of time axis can be catched easily. 
[BRIEF DESCRIPTION OF THE DRAWINGS] 

[FIGURE 1 ] Fig. 1 is a diagram for illustrating a whole configuration of 
contents-editing system; 

[ FIGURE 2 ] Fig. 2 is a block diagram showing a configuration of imaging apparatus; 

[FIGURE 3 ] Fig. 3 is a block diagram showing a configuration of another 
imaging apparatus; 

[FIGURE 4] Figs. 4 are diagrams showing relationship (part 1) between the 
image data and the associated information; 

[FIGURE 5 ] Figs. 5 are diagrams showing relationship (part 2) between the 
image data and the associated information; 

[FIGURE 6] Fig. 6 is a block diagram showing a configuration of an editing 
apparatus; 

[FIGURE 7 ] Fig. 7 is an illustration for GUI representation at an editing 
operation; 

[FIGURE 8] Fig. 8 is a flowchart showing a display control at the editing 
operation; 

[FIGURE 9] Figs. 9 are illustrations each for GUI representation at an editing 
operation; 

[FIGURE 10] Figs. 10 are illustrations showing a reproduction operation of 
the image; 

[FIGURE 1 1 ] Fig. 1 1 is a flowchart showing an editing process operation; and 
[ FIGURE 12] Fig. 1 2 is an illustration illustrating a configuration of a 

contents-editing system for editing contents by means of software. 
[EXPLANATION OF CODES] 

1 0, 1 0a Imaging apparatus 

12 Imaging unit 

13,17 Signal processor 
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402 Story board window 

403 Monitor viewer window 

404 Time line window 

405 Operation control portion 
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FIGURE 2 

Configuration of imaging apparatus 
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[SUMMARY] 

[OBJECT] The editing process is carried out easily using material created on standard 
frame rate and material varied frame rate. 

[MEANS FOR SOLVING] When an image is displayed on a material administration 
browser 401, a story board window 402 and a time line window 404 based on material 
data, the editing image display is controlled to display items of the material data that is 
combined with the frame rate information as the associated information and not combined 
with the frame rate information as the associated information with the items of the 
material data being distinguished from each other. When the material data is combined 
with the frame rate information as the associated information, a speed range available for 
reproduction of said material data is displayed based on the frame rate information. A 
console indication 403c in variable speeds for indicating a reproduction speed or for 
altering the reproduction speed within the speed range available for reproduction is 
provided. Thus, reproduced image may be displayed on material image representation 
403a at set reproduction speed. 
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